2023-11-24T06:11:53.138 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T06:11:53.138 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559 --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/orig.config.yaml 2023-11-24T06:11:53.185 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:11:53.268 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7465559' kernel: client: sha1: distro flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T06:11:53.268 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T06:11:53.269 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T06:11:53.269 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T06:11:53.271 INFO:teuthology.task.internal:Checking packages... 2023-11-24T06:11:53.295 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T06:11:53.295 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T06:11:53.295 INFO:teuthology.packaging:ref: None 2023-11-24T06:11:53.295 INFO:teuthology.packaging:tag: None 2023-11-24T06:11:53.295 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T06:11:53.296 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:11:53.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T06:11:53.539 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T06:11:53.539 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T06:11:53.546 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T06:11:53.546 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T06:11:53.558 INFO:teuthology.task.internal:Saving configuration 2023-11-24T06:11:53.575 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T06:11:53.581 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:11:53.607 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:06:23.689373', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:11:53.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:06:23.687360', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:11:53.632 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T06:11:53.638 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T06:11:53.638 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T06:11:53.638 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T06:11:53.710 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8cdd7a1b80>, signals=[15]) 2023-11-24T06:11:53.711 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T06:11:53.716 INFO:teuthology.task.internal:Opening connections... 2023-11-24T06:11:53.717 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:11:53.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:11:53.785 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:11:53.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:11:53.848 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T06:11:53.855 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2023-11-24T06:11:53.875 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2023-11-24T06:11:53.875 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-11-24T06:11:53.933 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T06:11:53.933 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T06:11:53.933 INFO:teuthology.orchestra.run.smithi114.stdout:ID="rhel" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="fedora" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8.6" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T06:11:53.934 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T06:11:53.935 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T06:11:53.935 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T06:11:53.935 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-11-24T06:11:53.959 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-11-24T06:11:53.979 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-11-24T06:11:53.979 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.6" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T06:11:54.035 INFO:teuthology.orchestra.run.smithi172.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T06:11:54.036 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T06:11:54.036 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-11-24T06:11:54.059 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T06:11:54.068 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T06:11:54.073 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T06:11:54.073 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:11:54.076 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:11:54.094 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T06:11:54.100 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T06:11:54.100 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:11:54.135 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:11:54.164 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T06:11:54.215 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T06:11:54.229 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T06:11:54.230 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-24T06:11:54.230 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-24T06:11:54.230 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:11:54.230 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:11:54.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:11:54.249 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2023-11-24T06:11:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:11:54.250 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2023-11-24T06:11:54.308 INFO:teuthology.orchestra.run.smithi172.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T06:11:54.308 INFO:teuthology.task.kernel:Running kernel on smithi172: 4.18.0-372.9.1.el8.x86_64 2023-11-24T06:11:54.308 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2023-11-24T06:11:54.310 INFO:teuthology.orchestra.run.smithi114.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T06:11:54.310 INFO:teuthology.task.kernel:Running kernel on smithi114: 4.18.0-372.9.1.el8.x86_64 2023-11-24T06:11:54.310 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2023-11-24T06:11:55.474 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-11-24T06:11:55.474 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-11-24T06:11:55.547 INFO:teuthology.orchestra.run.smithi114.stdout:Updating Subscription Management repositories. 2023-11-24T06:11:55.547 INFO:teuthology.orchestra.run.smithi114.stdout:Unable to read consumer identity 2023-11-24T06:11:56.998 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-11-24T06:11:57.020 INFO:teuthology.orchestra.run.smithi114.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T06:11:58.215 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-11-24T06:11:58.290 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-24T06:12:01.664 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 676 kB/s | 24 kB 00:00 2023-11-24T06:12:01.799 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 336 kB/s | 24 kB 00:00 2023-11-24T06:12:02.334 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T06:12:02.391 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2023-11-24T06:12:02.392 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2023-11-24T06:12:02.392 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2023-11-24T06:12:02.493 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T06:12:02.534 DEBUG:teuthology.orchestra.run.smithi114:> echo no | sudo yum reinstall kernel || true 2023-11-24T06:12:02.549 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-11-24T06:12:02.550 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2023-11-24T06:12:02.551 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-11-24T06:12:02.676 DEBUG:teuthology.orchestra.run.smithi172:> echo no | sudo yum reinstall kernel || true 2023-11-24T06:12:02.904 INFO:teuthology.orchestra.run.smithi114.stdout:Updating Subscription Management repositories. 2023-11-24T06:12:02.904 INFO:teuthology.orchestra.run.smithi114.stdout:Unable to read consumer identity 2023-11-24T06:12:03.047 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-11-24T06:12:03.047 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-11-24T06:12:03.162 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 06:12:01 AM UTC. 2023-11-24T06:12:03.202 INFO:teuthology.orchestra.run.smithi114.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T06:12:03.234 INFO:teuthology.orchestra.run.smithi114.stderr:Error: No packages marked for reinstall. 2023-11-24T06:12:03.280 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum reinstall -y kernel || true 2023-11-24T06:12:03.307 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 06:12:01 AM UTC. 2023-11-24T06:12:03.349 INFO:teuthology.orchestra.run.smithi172.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T06:12:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:Error: No packages marked for reinstall. 2023-11-24T06:12:03.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum reinstall -y kernel || true 2023-11-24T06:12:03.609 INFO:teuthology.orchestra.run.smithi114.stdout:Updating Subscription Management repositories. 2023-11-24T06:12:03.609 INFO:teuthology.orchestra.run.smithi114.stdout:Unable to read consumer identity 2023-11-24T06:12:03.760 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-11-24T06:12:03.760 INFO:teuthology.orchestra.run.smithi172.stdout:Unable to read consumer identity 2023-11-24T06:12:03.862 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 06:12:01 AM UTC. 2023-11-24T06:12:03.904 INFO:teuthology.orchestra.run.smithi114.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T06:12:03.932 INFO:teuthology.orchestra.run.smithi114.stderr:Error: No packages marked for reinstall. 2023-11-24T06:12:03.980 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T06:12:04.015 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:00:03 ago on Fri 24 Nov 2023 06:12:01 AM UTC. 2023-11-24T06:12:04.055 INFO:teuthology.orchestra.run.smithi172.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T06:12:04.058 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T06:12:04.059 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T06:12:04.059 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T06:12:04.059 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:04.090 INFO:teuthology.orchestra.run.smithi172.stderr:Error: No packages marked for reinstall. 2023-11-24T06:12:04.136 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T06:12:04.217 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T06:12:04.217 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T06:12:04.217 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T06:12:04.217 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:04.218 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T06:12:04.239 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T06:12:04.239 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:12:04.241 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:12:04.257 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T06:12:04.267 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T06:12:04.274 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T06:12:04.274 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:12:04.298 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:12:04.321 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T06:12:04.328 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T06:12:04.328 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:12:04.356 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:12:04.389 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:12:04.395 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:12:04.409 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:12:04.412 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:12:04.413 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T06:12:04.423 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T06:12:04.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:12:04.453 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:12:04.480 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T06:12:04.489 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T06:12:04.489 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:12:04.519 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:12:04.537 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:12:04.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:12:04.659 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:12:04.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:12:04.783 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:12:04.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:12:04.851 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:12:04.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:12:04.954 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:12:04.997 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:12:05.038 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:12:05.038 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:12:05.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2023-11-24T06:12:05.107 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2023-11-24T06:12:05.158 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:12:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:12:05.714 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T06:12:05.723 INFO:teuthology.task.internal:Starting timer... 2023-11-24T06:12:05.723 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T06:12:05.732 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T06:12:05.740 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-24T06:12:05.740 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2023-11-24T06:12:05.812 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2023-11-24T06:12:05.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2023-11-24T06:12:05.873 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2023-11-24T06:12:05.875 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T06:12:05.875 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2023-11-24T06:12:05.900 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2023-11-24T06:12:05.901 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2023-11-24T06:12:05.922 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2023-11-24T06:12:05.922 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2023-11-24T06:12:05.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:12:05.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:05.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:12:05.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:05.994 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T06:12:05.994 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:12:06.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:12:06.061 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T06:12:06.070 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-24T06:12:06.075 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-24T06:12:06.217 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T06:12:06.231 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T06:12:06.232 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:20:48.472 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2023-11-24T06:20:48.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-11-24T06:20:48.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:20:48.545 DEBUG:teuthology.orchestra.run.smithi114:> true 2023-11-24T06:20:48.611 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-11-24T06:20:48.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-11-24T06:20:48.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:20:48.681 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-11-24T06:20:48.748 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-11-24T06:20:48.748 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:20:48.761 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:20:48.761 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:20:48.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:20:48.764 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:20:48.764 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:20:48.794 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:20:48.809 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:20:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:20:48.838 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:20:48.890 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2023-11-24T06:20:48.901 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2023-11-24T06:20:48.915 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:20:48.915 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2023-11-24T06:20:48.927 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2023-11-24T06:20:48.929 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:20:48.942 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:20:48.955 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:20:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:bash: ntpq: command not found 2023-11-24T06:20:49.004 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:20:49.004 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2023-11-24T06:20:49.005 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.005 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.005 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.005 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.025 INFO:teuthology.orchestra.run.smithi172.stderr:bash: ntpq: command not found 2023-11-24T06:20:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:20:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2023-11-24T06:20:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:20:49.030 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:20:49.039 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:20:49.040 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:20:49.040 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:20:49.040 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:20:49.049 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:20:49.050 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:20:49.050 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:20:49.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:20:49.053 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:20:49.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:20:49.193 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T06:20:49.193 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:20:49.266 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T06:20:49.266 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:20:49.305 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:20:49.305 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:20:49.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:20:49.341 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:20:49.341 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:20:49.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:20:49.434 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:20:49.434 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:20:49.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:20:49.468 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:20:49.468 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:20:49.487 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:20:49.539 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:20:49.554 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2023-11-24T06:20:49.555 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2023-11-24T06:20:49.619 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:20:49.686 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2023-11-24T06:20:49.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2023-11-24T06:20:49.925 INFO:teuthology.orchestra.run.smithi114.stdout:Updating Subscription Management repositories. 2023-11-24T06:20:50.063 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-11-24T06:20:50.187 INFO:teuthology.orchestra.run.smithi114.stdout:44 files removed 2023-11-24T06:20:50.232 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:20:50.288 INFO:teuthology.orchestra.run.smithi172.stdout:44 files removed 2023-11-24T06:20:50.336 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:20:50.561 INFO:teuthology.orchestra.run.smithi114.stdout:Updating Subscription Management repositories. 2023-11-24T06:20:50.667 INFO:teuthology.orchestra.run.smithi172.stdout:Updating Subscription Management repositories. 2023-11-24T06:20:51.373 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 173 kB/s | 81 kB 00:00 2023-11-24T06:20:51.681 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 123 kB/s | 81 kB 00:00 2023-11-24T06:20:51.957 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 17 kB/s | 9.5 kB 00:00 2023-11-24T06:20:52.232 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 18 kB/s | 9.5 kB 00:00 2023-11-24T06:20:52.541 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-11-24T06:20:52.784 INFO:teuthology.orchestra.run.smithi114.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-11-24T06:20:52.891 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2023-11-24T06:20:53.100 INFO:teuthology.orchestra.run.smithi172.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-11-24T06:21:04.883 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 16 MB 00:11 2023-11-24T06:21:05.583 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 16 MB 00:12 2023-11-24T06:21:08.391 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 457 kB/s | 24 kB 00:00 2023-11-24T06:21:09.133 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 375 kB/s | 24 kB 00:00 2023-11-24T06:21:09.276 INFO:teuthology.orchestra.run.smithi172.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-11-24T06:21:10.010 INFO:teuthology.orchestra.run.smithi114.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 66 MB/s | 53 MB 00:00 2023-11-24T06:21:17.444 INFO:teuthology.orchestra.run.smithi172.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-11-24T06:21:18.086 INFO:teuthology.orchestra.run.smithi114.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-11-24T06:21:28.446 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:21:28.447 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:21:28.462 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T06:21:28.463 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T06:21:28.464 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T06:21:28.465 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T06:21:28.468 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T06:21:28.471 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T06:21:28.472 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T06:21:28.604 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2023-11-24T06:21:28.616 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-11-24T06:21:28.616 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2023-11-24T06:21:28.616 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T06:21:28.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T06:21:28.618 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T06:21:28.619 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T06:21:28.620 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T06:21:28.621 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T06:21:28.622 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T06:21:28.623 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T06:21:28.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T06:21:28.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T06:21:28.626 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T06:21:28.627 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T06:21:28.628 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout:Enabling module streams: 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql 10 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout:=========================================================================================================================== 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout:Install 128 Packages 2023-11-24T06:21:28.629 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 3 Packages 2023-11-24T06:21:28.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:21:28.630 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 209 M 2023-11-24T06:21:28.630 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2023-11-24T06:21:28.860 INFO:teuthology.orchestra.run.smithi172.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-11-24T06:21:28.992 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:21:28.993 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:21:29.008 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T06:21:29.009 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T06:21:29.010 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T06:21:29.011 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T06:21:29.014 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T06:21:29.017 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T06:21:29.018 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T06:21:29.102 INFO:teuthology.orchestra.run.smithi172.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 3.7 MB/s | 920 kB 00:00 2023-11-24T06:21:29.150 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2023-11-24T06:21:29.162 INFO:teuthology.orchestra.run.smithi114.stdout:=========================================================================================================================== 2023-11-24T06:21:29.162 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2023-11-24T06:21:29.162 INFO:teuthology.orchestra.run.smithi114.stdout:=========================================================================================================================== 2023-11-24T06:21:29.162 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T06:21:29.163 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T06:21:29.164 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T06:21:29.165 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T06:21:29.166 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T06:21:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T06:21:29.168 INFO:teuthology.orchestra.run.smithi114.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T06:21:29.169 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T06:21:29.170 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T06:21:29.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T06:21:29.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T06:21:29.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T06:21:29.174 INFO:teuthology.orchestra.run.smithi114.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T06:21:29.175 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling module streams: 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql 10 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout:=========================================================================================================================== 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout:Install 128 Packages 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 3 Packages 2023-11-24T06:21:29.176 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:21:29.177 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 209 M 2023-11-24T06:21:29.177 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2023-11-24T06:21:29.177 INFO:teuthology.orchestra.run.smithi172.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 2.2 MB/s | 171 kB 00:00 2023-11-24T06:21:29.478 INFO:teuthology.orchestra.run.smithi172.stdout:(4/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 7.7 MB/s | 2.3 MB 00:00 2023-11-24T06:21:29.653 INFO:teuthology.orchestra.run.smithi114.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-11-24T06:21:29.786 INFO:teuthology.orchestra.run.smithi172.stdout:(5/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 5.3 MB/s | 1.6 MB 00:00 2023-11-24T06:21:30.245 INFO:teuthology.orchestra.run.smithi114.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.5 MB/s | 920 kB 00:00 2023-11-24T06:21:30.279 INFO:teuthology.orchestra.run.smithi172.stdout:(6/131): ceph-common-18.0.0-7484.ga13ea5ff.el8. 15 MB/s | 23 MB 00:01 2023-11-24T06:21:30.320 INFO:teuthology.orchestra.run.smithi114.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 2.2 MB/s | 171 kB 00:00 2023-11-24T06:21:30.370 INFO:teuthology.orchestra.run.smithi172.stdout:(7/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 8.0 MB/s | 4.6 MB 00:00 2023-11-24T06:21:30.871 INFO:teuthology.orchestra.run.smithi172.stdout:(8/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 2.5 MB/s | 5.4 MB 00:02 2023-11-24T06:21:31.013 INFO:teuthology.orchestra.run.smithi172.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 171 kB/s | 24 kB 00:00 2023-11-24T06:21:31.562 INFO:teuthology.orchestra.run.smithi114.stdout:(4/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 2.4 MB/s | 5.4 MB 00:02 2023-11-24T06:21:31.604 INFO:teuthology.orchestra.run.smithi114.stdout:(5/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 1.8 MB/s | 2.3 MB 00:01 2023-11-24T06:21:31.887 INFO:teuthology.orchestra.run.smithi114.stdout:(6/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 5.0 MB/s | 1.6 MB 00:00 2023-11-24T06:21:31.929 INFO:teuthology.orchestra.run.smithi172.stdout:(10/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 11 MB/s | 18 MB 00:01 2023-11-24T06:21:32.004 INFO:teuthology.orchestra.run.smithi172.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 3.4 MB/s | 262 kB 00:00 2023-11-24T06:21:32.063 INFO:teuthology.orchestra.run.smithi172.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 532 kB/s | 31 kB 00:00 2023-11-24T06:21:32.205 INFO:teuthology.orchestra.run.smithi172.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 5.8 MB/s | 839 kB 00:00 2023-11-24T06:21:32.297 INFO:teuthology.orchestra.run.smithi172.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.0 MB/s | 183 kB 00:00 2023-11-24T06:21:32.373 INFO:teuthology.orchestra.run.smithi172.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 1.8 MB/s | 139 kB 00:00 2023-11-24T06:21:32.465 INFO:teuthology.orchestra.run.smithi172.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 5.5 MB/s | 517 kB 00:00 2023-11-24T06:21:32.748 INFO:teuthology.orchestra.run.smithi172.stdout:(17/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 5.8 MB/s | 14 MB 00:02 2023-11-24T06:21:32.823 INFO:teuthology.orchestra.run.smithi172.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 598 kB/s | 45 kB 00:00 2023-11-24T06:21:32.927 INFO:teuthology.orchestra.run.smithi172.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 1.3 MB/s | 128 kB 00:00 2023-11-24T06:21:33.024 INFO:teuthology.orchestra.run.smithi172.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.9 MB/s | 197 kB 00:00 2023-11-24T06:21:33.199 INFO:teuthology.orchestra.run.smithi172.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 2.1 MB/s | 371 kB 00:00 2023-11-24T06:21:33.349 INFO:teuthology.orchestra.run.smithi172.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 2.4 MB/s | 364 kB 00:00 2023-11-24T06:21:33.441 INFO:teuthology.orchestra.run.smithi172.stdout:(23/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 7.1 MB/s | 6.9 MB 00:00 2023-11-24T06:21:33.466 INFO:teuthology.orchestra.run.smithi172.stdout:(24/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 914 kB/s | 107 kB 00:00 2023-11-24T06:21:33.508 INFO:teuthology.orchestra.run.smithi172.stdout:(25/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-24T06:21:33.592 INFO:teuthology.orchestra.run.smithi172.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 2.1 MB/s | 181 kB 00:00 2023-11-24T06:21:33.651 INFO:teuthology.orchestra.run.smithi172.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 478 kB/s | 28 kB 00:00 2023-11-24T06:21:33.718 INFO:teuthology.orchestra.run.smithi172.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.9 MB/s | 133 kB 00:00 2023-11-24T06:21:33.787 INFO:teuthology.orchestra.run.smithi114.stdout:(7/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x86 9.5 MB/s | 18 MB 00:01 2023-11-24T06:21:34.001 INFO:teuthology.orchestra.run.smithi172.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 6.2 MB/s | 1.8 MB 00:00 2023-11-24T06:21:34.060 INFO:teuthology.orchestra.run.smithi172.stdout:(30/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 6.0 MB/s | 3.6 MB 00:00 2023-11-24T06:21:34.146 INFO:teuthology.orchestra.run.smithi114.stdout:(8/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 1.8 MB/s | 4.6 MB 00:02 2023-11-24T06:21:34.161 INFO:teuthology.orchestra.run.smithi172.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 2.4 MB/s | 243 kB 00:00 2023-11-24T06:21:34.238 INFO:teuthology.orchestra.run.smithi114.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 265 kB/s | 24 kB 00:00 2023-11-24T06:21:34.252 INFO:teuthology.orchestra.run.smithi172.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 547 kB/s | 50 kB 00:00 2023-11-24T06:21:34.353 INFO:teuthology.orchestra.run.smithi172.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 150 kB/s | 15 kB 00:00 2023-11-24T06:21:34.445 INFO:teuthology.orchestra.run.smithi172.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 2.8 MB/s | 262 kB 00:00 2023-11-24T06:21:34.628 INFO:teuthology.orchestra.run.smithi172.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 2.2 MB/s | 407 kB 00:00 2023-11-24T06:21:34.712 INFO:teuthology.orchestra.run.smithi172.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 217 kB/s | 18 kB 00:00 2023-11-24T06:21:34.904 INFO:teuthology.orchestra.run.smithi172.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-11-24T06:21:34.946 INFO:teuthology.orchestra.run.smithi172.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-11-24T06:21:34.996 INFO:teuthology.orchestra.run.smithi172.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-11-24T06:21:35.080 INFO:teuthology.orchestra.run.smithi172.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-11-24T06:21:35.163 INFO:teuthology.orchestra.run.smithi172.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 6.4 MB/s | 7.4 MB 00:01 2023-11-24T06:21:35.189 INFO:teuthology.orchestra.run.smithi172.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 401 kB/s | 44 kB 00:00 2023-11-24T06:21:35.238 INFO:teuthology.orchestra.run.smithi114.stdout:(10/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 9.6 MB/s | 14 MB 00:01 2023-11-24T06:21:35.248 INFO:teuthology.orchestra.run.smithi172.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 612 kB/s | 33 kB 00:00 2023-11-24T06:21:35.298 INFO:teuthology.orchestra.run.smithi172.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 328 kB/s | 16 kB 00:00 2023-11-24T06:21:35.338 INFO:teuthology.orchestra.run.smithi114.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.6 MB/s | 262 kB 00:00 2023-11-24T06:21:35.340 INFO:teuthology.orchestra.run.smithi172.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-11-24T06:21:35.390 INFO:teuthology.orchestra.run.smithi172.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-11-24T06:21:35.440 INFO:teuthology.orchestra.run.smithi172.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-11-24T06:21:35.447 INFO:teuthology.orchestra.run.smithi114.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 286 kB/s | 31 kB 00:00 2023-11-24T06:21:35.466 INFO:teuthology.orchestra.run.smithi172.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-11-24T06:21:35.491 INFO:teuthology.orchestra.run.smithi172.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-11-24T06:21:35.516 INFO:teuthology.orchestra.run.smithi172.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-11-24T06:21:35.575 INFO:teuthology.orchestra.run.smithi172.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2023-11-24T06:21:35.597 INFO:teuthology.orchestra.run.smithi114.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 5.5 MB/s | 839 kB 00:00 2023-11-24T06:21:35.600 INFO:teuthology.orchestra.run.smithi172.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 582 kB/s | 49 kB 00:00 2023-11-24T06:21:35.626 INFO:teuthology.orchestra.run.smithi172.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-11-24T06:21:35.651 INFO:teuthology.orchestra.run.smithi172.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-24T06:21:35.677 INFO:teuthology.orchestra.run.smithi172.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-24T06:21:35.714 INFO:teuthology.orchestra.run.smithi114.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.5 MB/s | 183 kB 00:00 2023-11-24T06:21:35.726 INFO:teuthology.orchestra.run.smithi172.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-11-24T06:21:35.752 INFO:teuthology.orchestra.run.smithi172.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-11-24T06:21:35.769 INFO:teuthology.orchestra.run.smithi172.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-11-24T06:21:35.819 INFO:teuthology.orchestra.run.smithi172.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2023-11-24T06:21:35.840 INFO:teuthology.orchestra.run.smithi114.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 1.1 MB/s | 139 kB 00:00 2023-11-24T06:21:35.844 INFO:teuthology.orchestra.run.smithi172.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-11-24T06:21:35.878 INFO:teuthology.orchestra.run.smithi172.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-11-24T06:21:35.903 INFO:teuthology.orchestra.run.smithi172.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2023-11-24T06:21:35.937 INFO:teuthology.orchestra.run.smithi172.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 472 kB/s | 27 kB 00:00 2023-11-24T06:21:35.954 INFO:teuthology.orchestra.run.smithi172.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-11-24T06:21:35.981 INFO:teuthology.orchestra.run.smithi114.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 3.6 MB/s | 517 kB 00:00 2023-11-24T06:21:36.054 INFO:teuthology.orchestra.run.smithi172.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-11-24T06:21:36.080 INFO:teuthology.orchestra.run.smithi172.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 488 kB/s | 61 kB 00:00 2023-11-24T06:21:36.105 INFO:teuthology.orchestra.run.smithi172.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-24T06:21:36.121 INFO:teuthology.orchestra.run.smithi172.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 551 kB/s | 23 kB 00:00 2023-11-24T06:21:36.147 INFO:teuthology.orchestra.run.smithi172.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-24T06:21:36.206 INFO:teuthology.orchestra.run.smithi172.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-11-24T06:21:36.231 INFO:teuthology.orchestra.run.smithi172.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 560 kB/s | 47 kB 00:00 2023-11-24T06:21:36.281 INFO:teuthology.orchestra.run.smithi172.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 963 kB/s | 43 kB 00:00 2023-11-24T06:21:36.307 INFO:teuthology.orchestra.run.smithi172.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-11-24T06:21:36.332 INFO:teuthology.orchestra.run.smithi172.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-24T06:21:36.357 INFO:teuthology.orchestra.run.smithi172.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2023-11-24T06:21:36.382 INFO:teuthology.orchestra.run.smithi172.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-24T06:21:36.408 INFO:teuthology.orchestra.run.smithi172.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 850 kB/s | 43 kB 00:00 2023-11-24T06:21:36.433 INFO:teuthology.orchestra.run.smithi172.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-11-24T06:21:36.458 INFO:teuthology.orchestra.run.smithi172.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-24T06:21:36.492 INFO:teuthology.orchestra.run.smithi172.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-11-24T06:21:36.517 INFO:teuthology.orchestra.run.smithi172.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-11-24T06:21:36.542 INFO:teuthology.orchestra.run.smithi172.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-24T06:21:36.576 INFO:teuthology.orchestra.run.smithi172.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-11-24T06:21:36.590 INFO:teuthology.orchestra.run.smithi114.stdout:(17/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 11 MB/s | 6.9 MB 00:00 2023-11-24T06:21:36.601 INFO:teuthology.orchestra.run.smithi172.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-11-24T06:21:36.635 INFO:teuthology.orchestra.run.smithi172.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-24T06:21:36.641 INFO:teuthology.orchestra.run.smithi114.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 891 kB/s | 45 kB 00:00 2023-11-24T06:21:36.669 INFO:teuthology.orchestra.run.smithi172.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-24T06:21:36.711 INFO:teuthology.orchestra.run.smithi172.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-11-24T06:21:36.716 INFO:teuthology.orchestra.run.smithi114.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 1.7 MB/s | 128 kB 00:00 2023-11-24T06:21:36.753 INFO:teuthology.orchestra.run.smithi172.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-24T06:21:36.795 INFO:teuthology.orchestra.run.smithi172.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-11-24T06:21:36.808 INFO:teuthology.orchestra.run.smithi114.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 2.1 MB/s | 197 kB 00:00 2023-11-24T06:21:36.820 INFO:teuthology.orchestra.run.smithi172.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 896 kB/s | 60 kB 00:00 2023-11-24T06:21:36.845 INFO:teuthology.orchestra.run.smithi172.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-11-24T06:21:36.871 INFO:teuthology.orchestra.run.smithi172.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-24T06:21:36.883 INFO:teuthology.orchestra.run.smithi114.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 4.8 MB/s | 371 kB 00:00 2023-11-24T06:21:36.896 INFO:teuthology.orchestra.run.smithi172.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-11-24T06:21:36.938 INFO:teuthology.orchestra.run.smithi172.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-11-24T06:21:36.958 INFO:teuthology.orchestra.run.smithi114.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 4.8 MB/s | 364 kB 00:00 2023-11-24T06:21:36.963 INFO:teuthology.orchestra.run.smithi172.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-11-24T06:21:36.988 INFO:teuthology.orchestra.run.smithi172.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-24T06:21:37.017 INFO:teuthology.orchestra.run.smithi114.stdout:(23/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 1.8 MB/s | 107 kB 00:00 2023-11-24T06:21:37.030 INFO:teuthology.orchestra.run.smithi172.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-24T06:21:37.064 INFO:teuthology.orchestra.run.smithi172.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-24T06:21:37.077 INFO:teuthology.orchestra.run.smithi114.stdout:(24/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-24T06:21:37.106 INFO:teuthology.orchestra.run.smithi172.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T06:21:37.148 INFO:teuthology.orchestra.run.smithi172.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-24T06:21:37.173 INFO:teuthology.orchestra.run.smithi172.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2023-11-24T06:21:37.232 INFO:teuthology.orchestra.run.smithi172.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-24T06:21:37.274 INFO:teuthology.orchestra.run.smithi172.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-11-24T06:21:37.307 INFO:teuthology.orchestra.run.smithi172.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-11-24T06:21:37.349 INFO:teuthology.orchestra.run.smithi172.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-24T06:21:37.401 INFO:teuthology.orchestra.run.smithi114.stdout:(25/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 11 MB/s | 3.6 MB 00:00 2023-11-24T06:21:37.468 INFO:teuthology.orchestra.run.smithi114.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 2.7 MB/s | 181 kB 00:00 2023-11-24T06:21:37.500 INFO:teuthology.orchestra.run.smithi172.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 26 MB/s | 14 MB 00:00 2023-11-24T06:21:37.543 INFO:teuthology.orchestra.run.smithi114.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 372 kB/s | 28 kB 00:00 2023-11-24T06:21:37.563 INFO:teuthology.orchestra.run.smithi172.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.3 MB/s | 713 kB 00:00 2023-11-24T06:21:37.600 INFO:teuthology.orchestra.run.smithi172.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 715 kB/s | 27 kB 00:00 2023-11-24T06:21:37.610 INFO:teuthology.orchestra.run.smithi114.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.9 MB/s | 133 kB 00:00 2023-11-24T06:21:37.642 INFO:teuthology.orchestra.run.smithi172.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T06:21:37.668 INFO:teuthology.orchestra.run.smithi172.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-11-24T06:21:37.734 INFO:teuthology.orchestra.run.smithi172.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-11-24T06:21:37.777 INFO:teuthology.orchestra.run.smithi172.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-11-24T06:21:37.794 INFO:teuthology.orchestra.run.smithi114.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 9.6 MB/s | 1.8 MB 00:00 2023-11-24T06:21:38.202 INFO:teuthology.orchestra.run.smithi172.stdout:(113/131): ceph-test-18.0.0-7484.ga13ea5ff.el8. 6.9 MB/s | 49 MB 00:07 2023-11-24T06:21:38.236 INFO:teuthology.orchestra.run.smithi172.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 491 kB/s | 225 kB 00:00 2023-11-24T06:21:38.269 INFO:teuthology.orchestra.run.smithi114.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 16 MB/s | 7.4 MB 00:00 2023-11-24T06:21:38.311 INFO:teuthology.orchestra.run.smithi172.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 5.8 MB/s | 3.7 MB 00:00 2023-11-24T06:21:38.336 INFO:teuthology.orchestra.run.smithi172.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 480 kB/s | 64 kB 00:00 2023-11-24T06:21:38.345 INFO:teuthology.orchestra.run.smithi114.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 3.2 MB/s | 243 kB 00:00 2023-11-24T06:21:38.361 INFO:teuthology.orchestra.run.smithi172.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-11-24T06:21:38.387 INFO:teuthology.orchestra.run.smithi172.stdout:(118/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 870 kB/s | 126 kB 00:00 2023-11-24T06:21:38.403 INFO:teuthology.orchestra.run.smithi114.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 863 kB/s | 50 kB 00:00 2023-11-24T06:21:38.412 INFO:teuthology.orchestra.run.smithi172.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.3 MB/s | 103 kB 00:00 2023-11-24T06:21:38.446 INFO:teuthology.orchestra.run.smithi172.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2023-11-24T06:21:38.462 INFO:teuthology.orchestra.run.smithi114.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 256 kB/s | 15 kB 00:00 2023-11-24T06:21:38.479 INFO:teuthology.orchestra.run.smithi172.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-11-24T06:21:38.521 INFO:teuthology.orchestra.run.smithi172.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-24T06:21:38.537 INFO:teuthology.orchestra.run.smithi114.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 3.4 MB/s | 262 kB 00:00 2023-11-24T06:21:38.563 INFO:teuthology.orchestra.run.smithi172.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-24T06:21:38.612 INFO:teuthology.orchestra.run.smithi114.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 5.3 MB/s | 407 kB 00:00 2023-11-24T06:21:38.647 INFO:teuthology.orchestra.run.smithi172.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2023-11-24T06:21:38.671 INFO:teuthology.orchestra.run.smithi114.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 310 kB/s | 18 kB 00:00 2023-11-24T06:21:38.722 INFO:teuthology.orchestra.run.smithi172.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-24T06:21:38.764 INFO:teuthology.orchestra.run.smithi172.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-11-24T06:21:38.823 INFO:teuthology.orchestra.run.smithi172.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-24T06:21:38.898 INFO:teuthology.orchestra.run.smithi172.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-11-24T06:21:38.930 INFO:teuthology.orchestra.run.smithi114.stdout:(37/131): ceph-common-18.0.0-7484.ga13ea5ff.el8 2.3 MB/s | 23 MB 00:09 2023-11-24T06:21:38.948 INFO:teuthology.orchestra.run.smithi172.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-24T06:21:39.063 INFO:teuthology.orchestra.run.smithi114.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 445 kB/s | 59 kB 00:00 2023-11-24T06:21:39.088 INFO:teuthology.orchestra.run.smithi114.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 734 kB/s | 306 kB 00:00 2023-11-24T06:21:39.130 INFO:teuthology.orchestra.run.smithi114.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-11-24T06:21:39.180 INFO:teuthology.orchestra.run.smithi114.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 877 kB/s | 44 kB 00:00 2023-11-24T06:21:39.256 INFO:teuthology.orchestra.run.smithi114.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.8 MB/s | 476 kB 00:00 2023-11-24T06:21:39.281 INFO:teuthology.orchestra.run.smithi114.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-11-24T06:21:39.307 INFO:teuthology.orchestra.run.smithi114.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-11-24T06:21:39.307 INFO:teuthology.orchestra.run.smithi172.stdout:(130/131): librados2-18.0.0-7484.ga13ea5ff.el8. 6.9 MB/s | 3.8 MB 00:00 2023-11-24T06:21:39.357 INFO:teuthology.orchestra.run.smithi172.stdout:(131/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 7.0 MB/s | 3.7 MB 00:00 2023-11-24T06:21:39.360 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:21:39.361 INFO:teuthology.orchestra.run.smithi172.stdout:Total 19 MB/s | 209 MB 00:10 2023-11-24T06:21:39.407 INFO:teuthology.orchestra.run.smithi114.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-11-24T06:21:39.558 INFO:teuthology.orchestra.run.smithi114.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 688 kB/s | 173 kB 00:00 2023-11-24T06:21:39.616 INFO:teuthology.orchestra.run.smithi114.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-11-24T06:21:39.641 INFO:teuthology.orchestra.run.smithi114.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-11-24T06:21:39.658 INFO:teuthology.orchestra.run.smithi114.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-11-24T06:21:39.675 INFO:teuthology.orchestra.run.smithi114.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 524 kB/s | 18 kB 00:00 2023-11-24T06:21:39.709 INFO:teuthology.orchestra.run.smithi114.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-11-24T06:21:39.743 INFO:teuthology.orchestra.run.smithi114.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-11-24T06:21:39.776 INFO:teuthology.orchestra.run.smithi114.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-11-24T06:21:39.885 INFO:teuthology.orchestra.run.smithi114.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-24T06:21:39.919 INFO:teuthology.orchestra.run.smithi114.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 470 kB/s | 16 kB 00:00 2023-11-24T06:21:39.952 INFO:teuthology.orchestra.run.smithi114.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-11-24T06:21:40.019 INFO:teuthology.orchestra.run.smithi114.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-11-24T06:21:40.061 INFO:teuthology.orchestra.run.smithi114.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-11-24T06:21:40.087 INFO:teuthology.orchestra.run.smithi114.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-11-24T06:21:40.112 INFO:teuthology.orchestra.run.smithi114.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-11-24T06:21:40.137 INFO:teuthology.orchestra.run.smithi114.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-11-24T06:21:40.192 INFO:teuthology.orchestra.run.smithi114.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 366 kB/s | 29 kB 00:00 2023-11-24T06:21:40.204 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2023-11-24T06:21:40.221 INFO:teuthology.orchestra.run.smithi114.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-11-24T06:21:40.296 INFO:teuthology.orchestra.run.smithi114.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.4 MB/s | 255 kB 00:00 2023-11-24T06:21:40.325 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2023-11-24T06:21:40.325 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2023-11-24T06:21:40.338 INFO:teuthology.orchestra.run.smithi114.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-11-24T06:21:40.397 INFO:teuthology.orchestra.run.smithi114.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-24T06:21:40.422 INFO:teuthology.orchestra.run.smithi114.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-11-24T06:21:40.447 INFO:teuthology.orchestra.run.smithi114.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-11-24T06:21:40.464 INFO:teuthology.orchestra.run.smithi114.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.2 MB/s | 70 kB 00:00 2023-11-24T06:21:40.523 INFO:teuthology.orchestra.run.smithi114.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-11-24T06:21:40.556 INFO:teuthology.orchestra.run.smithi114.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-11-24T06:21:40.601 INFO:teuthology.orchestra.run.smithi114.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-11-24T06:21:40.641 INFO:teuthology.orchestra.run.smithi114.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-11-24T06:21:40.682 INFO:teuthology.orchestra.run.smithi114.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-11-24T06:21:40.725 INFO:teuthology.orchestra.run.smithi114.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-11-24T06:21:40.766 INFO:teuthology.orchestra.run.smithi114.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-11-24T06:21:40.809 INFO:teuthology.orchestra.run.smithi114.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-11-24T06:21:40.850 INFO:teuthology.orchestra.run.smithi114.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-11-24T06:21:40.892 INFO:teuthology.orchestra.run.smithi114.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 13 MB/s | 560 kB 00:00 2023-11-24T06:21:40.934 INFO:teuthology.orchestra.run.smithi114.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 451 kB/s | 19 kB 00:00 2023-11-24T06:21:40.968 INFO:teuthology.orchestra.run.smithi114.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 4.5 MB/s | 155 kB 00:00 2023-11-24T06:21:41.010 INFO:teuthology.orchestra.run.smithi114.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-11-24T06:21:41.044 INFO:teuthology.orchestra.run.smithi114.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 536 kB/s | 18 kB 00:00 2023-11-24T06:21:41.094 INFO:teuthology.orchestra.run.smithi114.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-11-24T06:21:41.136 INFO:teuthology.orchestra.run.smithi114.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-11-24T06:21:41.236 INFO:teuthology.orchestra.run.smithi114.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 23 MB/s | 2.3 MB 00:00 2023-11-24T06:21:41.278 INFO:teuthology.orchestra.run.smithi114.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-11-24T06:21:41.320 INFO:teuthology.orchestra.run.smithi114.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-24T06:21:41.362 INFO:teuthology.orchestra.run.smithi114.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-11-24T06:21:41.404 INFO:teuthology.orchestra.run.smithi114.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-24T06:21:41.438 INFO:teuthology.orchestra.run.smithi114.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-11-24T06:21:41.470 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2023-11-24T06:21:41.471 INFO:teuthology.orchestra.run.smithi114.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-11-24T06:21:41.474 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2023-11-24T06:21:41.513 INFO:teuthology.orchestra.run.smithi114.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 15 MB/s | 644 kB 00:00 2023-11-24T06:21:41.555 INFO:teuthology.orchestra.run.smithi114.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2023-11-24T06:21:41.597 INFO:teuthology.orchestra.run.smithi114.stdout:(95/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-11-24T06:21:41.857 INFO:teuthology.orchestra.run.smithi114.stdout:(96/131): python3-scipy-1.0.0-21.module+el8.5.0 53 MB/s | 14 MB 00:00 2023-11-24T06:21:41.898 INFO:teuthology.orchestra.run.smithi114.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-24T06:21:41.931 INFO:teuthology.orchestra.run.smithi114.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-24T06:21:41.973 INFO:teuthology.orchestra.run.smithi114.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T06:21:42.015 INFO:teuthology.orchestra.run.smithi114.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-24T06:21:42.049 INFO:teuthology.orchestra.run.smithi114.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-11-24T06:21:42.091 INFO:teuthology.orchestra.run.smithi114.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 16 MB/s | 688 kB 00:00 2023-11-24T06:21:42.133 INFO:teuthology.orchestra.run.smithi114.stdout:(103/131): thrift-0.13.0-2.el8.x86_64.rpm 1.0 MB/s | 1.7 MB 00:01 2023-11-24T06:21:42.167 INFO:teuthology.orchestra.run.smithi114.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-11-24T06:21:42.249 INFO:teuthology.orchestra.run.smithi114.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2023-11-24T06:21:42.276 INFO:teuthology.orchestra.run.smithi114.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-11-24T06:21:42.309 INFO:teuthology.orchestra.run.smithi114.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-11-24T06:21:42.343 INFO:teuthology.orchestra.run.smithi114.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 801 kB/s | 27 kB 00:00 2023-11-24T06:21:42.385 INFO:teuthology.orchestra.run.smithi114.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T06:21:42.418 INFO:teuthology.orchestra.run.smithi114.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-24T06:21:42.485 INFO:teuthology.orchestra.run.smithi114.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-11-24T06:21:42.527 INFO:teuthology.orchestra.run.smithi114.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-11-24T06:21:42.561 INFO:teuthology.orchestra.run.smithi114.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 6.6 MB/s | 225 kB 00:00 2023-11-24T06:21:42.619 INFO:teuthology.orchestra.run.smithi114.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-11-24T06:21:42.645 INFO:teuthology.orchestra.run.smithi114.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-11-24T06:21:42.670 INFO:teuthology.orchestra.run.smithi114.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-24T06:21:42.695 INFO:teuthology.orchestra.run.smithi114.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-11-24T06:21:42.712 INFO:teuthology.orchestra.run.smithi114.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-11-24T06:21:42.746 INFO:teuthology.orchestra.run.smithi114.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-11-24T06:21:42.788 INFO:teuthology.orchestra.run.smithi114.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-24T06:21:42.880 INFO:teuthology.orchestra.run.smithi114.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-11-24T06:21:42.922 INFO:teuthology.orchestra.run.smithi114.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-24T06:21:42.997 INFO:teuthology.orchestra.run.smithi114.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-11-24T06:21:43.031 INFO:teuthology.orchestra.run.smithi114.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-11-24T06:21:43.089 INFO:teuthology.orchestra.run.smithi114.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-11-24T06:21:43.173 INFO:teuthology.orchestra.run.smithi114.stdout:(126/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-11-24T06:21:43.206 INFO:teuthology.orchestra.run.smithi114.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-11-24T06:21:43.940 INFO:teuthology.orchestra.run.smithi114.stdout:(128/131): librados2-18.0.0-7484.ga13ea5ff.el8. 4.9 MB/s | 3.8 MB 00:00 2023-11-24T06:21:43.999 INFO:teuthology.orchestra.run.smithi114.stdout:(129/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 4.7 MB/s | 3.7 MB 00:00 2023-11-24T06:21:44.041 INFO:teuthology.orchestra.run.smithi114.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 7.8 MB/s | 798 kB 00:00 2023-11-24T06:21:46.158 INFO:teuthology.orchestra.run.smithi114.stdout:(131/131): ceph-test-18.0.0-7484.ga13ea5ff.el8. 4.1 MB/s | 49 MB 00:11 2023-11-24T06:21:46.162 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:21:46.163 INFO:teuthology.orchestra.run.smithi114.stdout:Total 12 MB/s | 209 MB 00:16 2023-11-24T06:21:46.262 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2023-11-24T06:21:46.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T06:21:46.617 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T06:21:46.786 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:21:46.930 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:21:46.991 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2023-11-24T06:21:47.111 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2023-11-24T06:21:47.112 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2023-11-24T06:21:47.128 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:21:47.534 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:21:47.761 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:21:48.155 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:21:48.238 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2023-11-24T06:21:48.243 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2023-11-24T06:21:48.352 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:21:48.479 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:21:48.677 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:21:48.770 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:21:49.154 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T06:21:49.398 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T06:21:49.605 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T06:21:50.005 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T06:21:50.242 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T06:21:50.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:21:50.523 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:21:50.694 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:21:50.822 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:21:51.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T06:21:51.191 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T06:21:51.467 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T06:21:51.610 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:21:51.744 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:21:51.902 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:21:52.507 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:21:52.668 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:21:53.271 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:21:53.394 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2023-11-24T06:21:53.451 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:21:53.593 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T06:21:53.785 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T06:21:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:21:54.073 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:21:54.156 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:21:54.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:21:54.446 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T06:21:54.657 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T06:21:54.668 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:21:54.896 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T06:21:54.903 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:21:55.066 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T06:21:55.196 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T06:21:55.318 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:21:55.456 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T06:21:55.561 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:21:55.651 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:21:55.679 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:21:56.111 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:21:56.197 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:21:56.375 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:21:56.606 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T06:21:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T06:21:57.049 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T06:21:57.417 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:21:57.442 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T06:21:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T06:21:57.886 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:21:57.981 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:21:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:21:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:21:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T06:21:58.464 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T06:21:58.650 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T06:21:58.766 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T06:21:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T06:21:58.936 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:21:59.028 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T06:21:59.079 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:21:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:21:59.229 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T06:21:59.446 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T06:21:59.631 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T06:21:59.789 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T06:21:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:21:59.949 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T06:21:59.969 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:22:00.100 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T06:22:00.253 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T06:22:00.417 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:22:00.488 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:22:00.577 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:22:00.695 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T06:22:00.785 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:22:00.920 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T06:22:01.059 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:22:01.125 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:22:01.354 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T06:22:01.531 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T06:22:01.532 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:22:01.695 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T06:22:01.885 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T06:22:01.934 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T06:22:02.069 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T06:22:02.108 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T06:22:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T06:22:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T06:22:02.520 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T06:22:02.596 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T06:22:02.708 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T06:22:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T06:22:02.891 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T06:22:03.027 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:22:03.260 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T06:22:03.518 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T06:22:03.727 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T06:22:03.970 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T06:22:04.213 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T06:22:04.541 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T06:22:04.774 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T06:22:04.790 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:22:04.942 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T06:22:05.144 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T06:22:05.332 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T06:22:05.511 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T06:22:05.782 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T06:22:05.798 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T06:22:06.054 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T06:22:06.116 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T06:22:06.312 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T06:22:06.353 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T06:22:06.534 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T06:22:06.565 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T06:22:06.673 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T06:22:06.763 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T06:22:06.852 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T06:22:06.971 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T06:22:07.116 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T06:22:07.140 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T06:22:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T06:22:07.314 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T06:22:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T06:22:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T06:22:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T06:22:07.601 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T06:22:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:22:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:22:07.806 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T06:22:08.018 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T06:22:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T06:22:08.209 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T06:22:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T06:22:08.368 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T06:22:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:22:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:22:08.504 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T06:22:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T06:22:08.707 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T06:22:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T06:22:08.919 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T06:22:09.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T06:22:09.096 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T06:22:09.228 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T06:22:09.340 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T06:22:09.379 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T06:22:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T06:22:09.556 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T06:22:09.681 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T06:22:09.713 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T06:22:09.862 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T06:22:09.921 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T06:22:10.217 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T06:22:10.494 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T06:22:10.687 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T06:22:10.930 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T06:22:11.039 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T06:22:11.147 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T06:22:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T06:22:11.434 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T06:22:11.515 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T06:22:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T06:22:11.800 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T06:22:11.833 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T06:22:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T06:22:12.079 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T06:22:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T06:22:12.278 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T06:22:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T06:22:12.723 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T06:22:12.973 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T06:22:13.084 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T06:22:13.189 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T06:22:13.265 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:22:13.345 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:22:13.418 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T06:22:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T06:22:13.583 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T06:22:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T06:22:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T06:22:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T06:22:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T06:22:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T06:22:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T06:22:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T06:22:15.159 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T06:22:15.311 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T06:22:15.430 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T06:22:15.624 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T06:22:15.836 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T06:22:16.047 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T06:22:16.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T06:22:16.507 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T06:22:16.689 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T06:22:16.914 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T06:22:17.204 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:17.423 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T06:22:18.019 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:18.287 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T06:22:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T06:22:18.601 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:22:18.686 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:22:18.801 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T06:22:18.830 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:22:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T06:22:19.288 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T06:22:20.082 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T06:22:20.274 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:22:20.354 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:22:20.534 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T06:22:24.163 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:24.391 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:24.965 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:22:25.288 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:22:25.365 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:22:25.524 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:22:37.996 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2023-11-24T06:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2023-11-24T06:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-11-24T06:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /run 2023-11-24T06:22:37.997 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2023-11-24T06:22:37.998 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2023-11-24T06:22:37.998 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2023-11-24T06:22:37.998 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2023-11-24T06:22:37.998 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /dev 2023-11-24T06:22:37.998 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:22:38.344 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:22:39.946 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:22:40.122 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:22:40.351 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:22:40.494 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:22:40.950 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:22:41.135 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:22:41.152 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:22:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:22:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:22:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T06:22:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T06:22:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:22:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /dev 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /run 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2023-11-24T06:22:43.952 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2023-11-24T06:22:43.953 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2023-11-24T06:22:43.953 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2023-11-24T06:22:43.953 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /dev 2023-11-24T06:22:43.953 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:22:44.079 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:22:44.263 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:22:45.040 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:22:45.192 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:22:45.285 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:22:45.638 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T06:22:45.821 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:22:45.858 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:22:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:22:46.042 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:22:46.054 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:22:46.060 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:22:46.286 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:22:46.447 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:22:46.902 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:22:47.113 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:22:47.129 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:22:47.383 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T06:22:47.595 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:22:47.629 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:22:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:22:47.827 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:22:47.968 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:22:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:22:48.501 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:22:48.781 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:22:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:22:49.086 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T06:22:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T06:22:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:22:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:22:49.960 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:22:50.924 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:22:51.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:22:51.143 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:22:51.536 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T06:22:51.717 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:22:51.754 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:22:51.928 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:22:51.940 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:22:53.182 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:22:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T06:22:53.469 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:22:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:22:53.654 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:22:53.696 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:22:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:22:54.070 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T06:22:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:22:54.379 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:22:54.395 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T06:22:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T06:22:54.894 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T06:22:55.040 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:22:55.235 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:22:55.437 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T06:22:55.620 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T06:22:55.777 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T06:22:57.954 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:22:58.182 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:22:58.201 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:22:58.341 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T06:22:58.648 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T06:22:58.960 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T06:22:59.065 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:22:59.306 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T06:22:59.457 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:22:59.535 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:22:59.723 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:22:59.736 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:22:59.873 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:22:59.882 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:22:59.923 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T06:23:00.009 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T06:23:00.009 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:00.193 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:00.193 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:23:00.267 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T06:23:00.463 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:23:00.479 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T06:23:00.647 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:00.706 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T06:23:00.852 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:23:01.050 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T06:23:01.157 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T06:23:01.158 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T06:23:01.159 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T06:23:01.160 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T06:23:01.161 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T06:23:01.162 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T06:23:01.163 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T06:23:01.164 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T06:23:01.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T06:23:01.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T06:23:01.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T06:23:01.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T06:23:01.165 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T06:23:01.166 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T06:23:01.167 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T06:23:01.168 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T06:23:01.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:01.169 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T06:23:01.290 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T06:23:01.464 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T06:23:01.595 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T06:23:03.782 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:03.782 INFO:teuthology.orchestra.run.smithi172.stdout:Last metadata expiration check: 0:01:47 ago on Fri 24 Nov 2023 06:21:16 AM UTC. 2023-11-24T06:23:03.788 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:23:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:23:04.054 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:23:04.202 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T06:23:04.508 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T06:23:04.790 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T06:23:05.151 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T06:23:05.326 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:23:05.408 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:23:05.602 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:23:05.614 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:23:05.768 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:23:05.776 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:23:05.920 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T06:23:05.921 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:06.121 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:06.121 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:23:06.339 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:23:06.375 INFO:teuthology.orchestra.run.smithi172.stdout:Installed products updated. 2023-11-24T06:23:06.550 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:06.615 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:06.615 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2023-11-24T06:23:06.615 INFO:teuthology.orchestra.run.smithi172.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T06:23:06.615 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.615 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.616 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.617 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: jq-1.6-3.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.618 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T06:23:06.619 INFO:teuthology.orchestra.run.smithi172.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T06:23:06.620 INFO:teuthology.orchestra.run.smithi172.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T06:23:06.621 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T06:23:06.622 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T06:23:06.623 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T06:23:06.624 INFO:teuthology.orchestra.run.smithi172.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:06.625 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T06:23:06.626 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:06.627 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2023-11-24T06:23:06.793 DEBUG:teuthology.parallel:result is None 2023-11-24T06:23:07.051 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:07.051 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T06:23:07.051 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T06:23:07.051 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T06:23:07.052 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T06:23:07.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T06:23:07.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T06:23:07.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T06:23:07.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T06:23:07.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T06:23:07.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T06:23:07.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T06:23:07.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T06:23:07.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T06:23:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T06:23:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T06:23:07.070 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T06:23:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T06:23:07.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:23:07.073 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T06:23:09.755 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:23:09.756 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:52 ago on Fri 24 Nov 2023 06:21:17 AM UTC. 2023-11-24T06:23:12.327 INFO:teuthology.orchestra.run.smithi114.stdout:Installed products updated. 2023-11-24T06:23:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:12.554 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2023-11-24T06:23:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T06:23:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: jq-1.6-3.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T06:23:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T06:23:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T06:23:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T06:23:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T06:23:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T06:23:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T06:23:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T06:23:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T06:23:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:23:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T06:23:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T06:23:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T06:23:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T06:23:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T06:23:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T06:23:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:12.568 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2023-11-24T06:23:12.832 DEBUG:teuthology.parallel:result is None 2023-11-24T06:23:12.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:23:13.050 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:23:13.074 INFO:teuthology.orchestra.run.smithi114.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:23:13.075 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:23:13.075 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:23:13.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:23:13.215 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:23:13.244 INFO:teuthology.orchestra.run.smithi172.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:23:13.244 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:23:13.244 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:23:13.245 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:23:13.245 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:13.245 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:23:13.286 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:13.287 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:23:13.334 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:23:13.334 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:13.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:23:13.361 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:23:13.433 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:13.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:23:13.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:23:13.534 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:23:13.534 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:13.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:23:13.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:23:13.629 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:13.630 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:23:13.657 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:23:13.725 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:23:13.726 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:13.726 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:23:13.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:23:13.820 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:13.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:23:13.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:23:13.915 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:23:13.984 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:23:13.984 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2023-11-24T06:23:13.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2023-11-24T06:23:14.013 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:23:14.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:23:14.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:23:14.081 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:23:14.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:23:14.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:23:14.203 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:23:14.203 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:23:14.204 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:23:14.204 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:14.204 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:23:14.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:14.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:14.375 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:14.375 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:23:14.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:14.777 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:14.812 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:23:14.812 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:23:14.812 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7465559', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:23:14.813 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:23:14.816 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:23:14.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:23:14.892 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:23:14.959 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:14.959 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:23:14.977 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:23:14.977 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2023-11-24T06:23:15.035 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:23:15.035 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.035 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 385791 Links: 1 2023-11-24T06:23:15.035 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.035 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.036 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2023-11-24 06:23:06.755609914 +0000 2023-11-24T06:23:15.036 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2023-11-24 06:20:13.048943014 +0000 2023-11-24T06:23:15.036 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2023-11-24 06:20:13.048943014 +0000 2023-11-24T06:23:15.036 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2023-11-24T06:23:15.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:23:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2023-11-24T06:23:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2023-11-24T06:23:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000263 s, 1.9 MB/s 2023-11-24T06:23:15.103 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:23:15.161 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2023-11-24T06:23:15.219 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 388929 Links: 1 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2023-11-24 06:23:06.755609914 +0000 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2023-11-24 06:20:13.533932003 +0000 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2023-11-24 06:20:13.533932003 +0000 2023-11-24T06:23:15.220 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2023-11-24T06:23:15.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:23:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2023-11-24T06:23:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2023-11-24T06:23:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000242515 s, 2.1 MB/s 2023-11-24T06:23:15.287 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:23:15.346 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 389622 Links: 1 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.403 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2023-11-24 06:23:06.755609914 +0000 2023-11-24T06:23:15.404 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2023-11-24 06:20:14.034920628 +0000 2023-11-24T06:23:15.404 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2023-11-24 06:20:14.034920628 +0000 2023-11-24T06:23:15.404 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2023-11-24T06:23:15.404 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:23:15.468 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2023-11-24T06:23:15.468 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2023-11-24T06:23:15.468 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000204162 s, 2.5 MB/s 2023-11-24T06:23:15.469 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:23:15.527 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 391576 Links: 1 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.584 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2023-11-24 06:23:06.755609914 +0000 2023-11-24T06:23:15.585 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2023-11-24 06:20:14.533909299 +0000 2023-11-24T06:23:15.585 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2023-11-24 06:20:14.533909299 +0000 2023-11-24T06:23:15.585 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2023-11-24T06:23:15.585 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:23:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2023-11-24T06:23:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2023-11-24T06:23:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000178077 s, 2.9 MB/s 2023-11-24T06:23:15.650 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:23:15.708 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:15.708 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:15.708 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:23:15.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:23:15.729 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 390387 Links: 1 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.790 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-11-24 06:23:00.843754732 +0000 2023-11-24T06:23:15.791 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-11-24 06:20:19.150171920 +0000 2023-11-24T06:23:15.791 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-11-24 06:20:19.150171920 +0000 2023-11-24T06:23:15.791 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-11-24T06:23:15.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:23:15.861 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-11-24T06:23:15.861 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-11-24T06:23:15.861 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000342744 s, 1.5 MB/s 2023-11-24T06:23:15.862 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:23:15.923 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2023-11-24T06:23:15.983 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:23:15.983 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:15.983 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 390466 Links: 1 2023-11-24T06:23:15.983 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:15.983 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:15.984 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-11-24 06:23:00.843754732 +0000 2023-11-24T06:23:15.984 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-11-24 06:20:19.663159445 +0000 2023-11-24T06:23:15.984 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-11-24 06:20:19.663159445 +0000 2023-11-24T06:23:15.984 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-11-24T06:23:15.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:23:16.053 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-11-24T06:23:16.054 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-11-24T06:23:16.054 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00024042 s, 2.1 MB/s 2023-11-24T06:23:16.055 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:23:16.116 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2023-11-24T06:23:16.175 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:23:16.176 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:16.176 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 382946 Links: 1 2023-11-24T06:23:16.176 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:16.176 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:16.177 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-11-24 06:23:00.843754732 +0000 2023-11-24T06:23:16.177 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-11-24 06:20:20.171147091 +0000 2023-11-24T06:23:16.177 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-11-24 06:20:20.171147091 +0000 2023-11-24T06:23:16.177 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-11-24T06:23:16.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:23:16.244 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-11-24T06:23:16.245 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-11-24T06:23:16.245 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000240564 s, 2.1 MB/s 2023-11-24T06:23:16.246 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:23:16.305 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2023-11-24T06:23:16.363 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 393630 Links: 1 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2023-11-24 06:23:00.843754732 +0000 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2023-11-24 06:20:20.672134908 +0000 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2023-11-24 06:20:20.672134908 +0000 2023-11-24T06:23:16.364 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2023-11-24T06:23:16.365 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:23:16.431 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2023-11-24T06:23:16.432 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2023-11-24T06:23:16.432 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000207294 s, 2.5 MB/s 2023-11-24T06:23:16.433 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:23:16.492 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:16.493 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:23:16.493 INFO:tasks.ceph:Generating config... 2023-11-24T06:23:16.494 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:23:16.494 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:23:16.494 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:23:16.495 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:23:16.495 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:23:16.495 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:23:16.495 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-24T06:23:16.495 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:23:16.495 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:23:16.495 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:23:16.495 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:23:16.496 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:23:16.497 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:23:16.497 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:23:16.497 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:23:16.497 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:23:16.497 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:23:16.498 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:23:16.499 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:23:16.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:23:16.587 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:23:16.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:23:16.638 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:23:16.666 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.114'), ('mon.b', '172.21.15.172'), ('mon.c', '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]')] 2023-11-24T06:23:16.666 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.114,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:23:16.667 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:16.667 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:23:16.723 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.114 --add b 172.21.15.172 --addv c '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:16.813 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: generated fsid cfb0acb2-02f2-43f9-bba4-c35657dcf6fd 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = pacific 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 0 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:fsid cfb0acb2-02f2-43f9-bba4-c35657dcf6fd 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:last_changed 2023-11-24T06:23:16.811943+0000 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:created 2023-11-24T06:23:16.811943+0000 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:min_mon_release 16 (pacific) 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:election_strategy: 1 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2023-11-24T06:23:16.814 INFO:teuthology.orchestra.run.smithi114.stdout:2: [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] mon.c 2023-11-24T06:23:16.815 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:23:16.816 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:23:16.832 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cfb0acb2-02f2-43f9-bba4-c35657dcf6fd... 2023-11-24T06:23:16.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:23:16.874 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:23:16.917 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2023-11-24T06:23:16.918 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2023-11-24T06:23:16.918 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:23:16.918 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2023-11-24T06:23:16.918 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.918 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2023-11-24T06:23:16.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.919 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2023-11-24T06:23:16.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.919 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2023-11-24T06:23:16.920 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2023-11-24T06:23:16.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.920 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2023-11-24T06:23:16.920 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2023-11-24T06:23:16.921 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.921 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:23:16.921 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2023-11-24T06:23:16.921 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.921 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2023-11-24T06:23:16.922 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2023-11-24T06:23:16.923 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:23:16.923 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:23:16.923 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2023-11-24T06:23:16.923 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:23:16.923 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:23:16.924 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.114,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-11-24T06:23:16.925 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = cfb0acb2-02f2-43f9-bba4-c35657dcf6fd 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.926 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2023-11-24T06:23:16.927 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2023-11-24T06:23:16.928 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: bdev async discard = True 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: bdev enable discard = True 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore allocator = bitmap 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 1/20 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 1/20 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 4/10 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:23:16.929 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2023-11-24T06:23:16.930 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug finisher = 20 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2023-11-24T06:23:16.931 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:23:16.932 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2023-11-24T06:23:16.933 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = "" 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2023-11-24T06:23:16.934 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = True 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = True 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2023-11-24T06:23:16.935 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2023-11-24T06:23:16.936 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2023-11-24T06:23:16.936 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2023-11-24T06:23:16.936 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:23:16.936 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2023-11-24T06:23:16.937 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2023-11-24T06:23:16.938 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.939 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.114,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = cfb0acb2-02f2-43f9-bba4-c35657dcf6fd 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.940 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2023-11-24T06:23:16.941 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.942 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bdev async discard = True 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bdev enable discard = True 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore allocator = bitmap 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2023-11-24T06:23:16.943 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 1/20 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 1/20 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 4/10 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-11-24T06:23:16.944 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2023-11-24T06:23:16.945 INFO:teuthology.orchestra.run.smithi172.stdout: debug finisher = 20 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:23:16.946 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:23:16.947 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2023-11-24T06:23:16.948 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2023-11-24T06:23:16.949 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2023-11-24T06:23:16.950 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2023-11-24T06:23:16.951 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2023-11-24T06:23:16.953 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:23:16.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:23:17.050 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:23:17.051 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:17.051 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:23:17.067 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:17.067 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:23:17.123 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:23:17.123 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:17.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:23:17.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:23:17.199 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:17.199 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:17.256 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:23:17.256 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:17.256 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:23:17.256 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:23:17.298 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:17.299 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:17.357 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:23:17.357 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:23:17.357 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:23:17.410 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:23:17.413 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:23:17.495 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:23:17.498 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:23:17.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:23:17.554 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:23:17.555 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:23:17.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:23:17.674 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:23:17.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:23:17.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:23:17.760 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:23:17.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:23:17.789 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:23:17.882 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:23:17.885 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:23:17.911 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:23:17.911 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:23:17.955 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:23:17.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:23:18.052 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:23:18.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:23:18.108 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:23:18.124 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:23:18.207 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:23:18.220 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:23:18.220 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:23:18.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:23:18.247 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:18.247 INFO:tasks.ceph:role: osd.0 2023-11-24T06:23:18.247 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:23:18.247 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:23:18.320 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:18.321 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:18.321 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:18.321 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:18.321 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:18.322 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:18.322 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:18.756 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2023-11-24T06:23:18.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2023-11-24T06:23:18.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:23:18.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:23:18.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:23:19.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:19.035 INFO:tasks.ceph:role: osd.1 2023-11-24T06:23:19.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:23:19.036 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:23:19.111 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:19.112 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:19.533 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2023-11-24T06:23:19.535 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2023-11-24T06:23:19.536 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:23:19.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:23:19.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:23:19.750 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:19.751 INFO:tasks.ceph:role: osd.2 2023-11-24T06:23:19.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:23:19.751 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:20.271 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2023-11-24T06:23:20.273 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2023-11-24T06:23:20.274 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:23:20.457 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:23:20.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:23:20.563 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:20.563 INFO:tasks.ceph:role: osd.3 2023-11-24T06:23:20.563 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:23:20.563 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:20.639 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:21.073 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2023-11-24T06:23:21.075 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2023-11-24T06:23:21.075 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:23:21.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:23:21.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:21.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:23:21.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:23:21.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:23:21.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:23:21.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:23:21.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:21.400+0000 7f2b44ff4640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:23:24.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:23:24.300 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.389+0000 7f730eebf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.390+0000 7f730eebf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.390+0000 7f730eebf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.390+0000 7f730eebf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.390+0000 7f730eebf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:23:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:24.390+0000 7f730eebf640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:23:27.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:23:27.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:27.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:23:27.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:23:27.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:23:27.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:23:27.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:23:27.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:27.408+0000 7f19c2fa6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:23:30.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:23:30.912 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.003+0000 7f73d5c27640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:23:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.004+0000 7f73d5c27640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:23:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.004+0000 7f73d5c27640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:23:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.004+0000 7f73d5c27640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:23:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.004+0000 7f73d5c27640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:23:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:31.004+0000 7f73d5c27640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:23:33.884 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:23:33.916 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:23:33.952 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:33.952 INFO:tasks.ceph:role: osd.4 2023-11-24T06:23:33.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:23:33.952 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:23:34.030 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:34.030 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:34.030 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:34.030 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:34.031 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:34.460 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-11-24T06:23:34.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-11-24T06:23:34.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:23:34.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:23:34.678 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:23:34.748 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:34.748 INFO:tasks.ceph:role: osd.5 2023-11-24T06:23:34.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:23:34.748 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:23:34.824 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:34.825 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:34.826 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:34.826 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:35.249 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-11-24T06:23:35.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-11-24T06:23:35.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:23:35.367 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:23:35.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:23:35.469 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:35.469 INFO:tasks.ceph:role: osd.6 2023-11-24T06:23:35.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:23:35.469 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:23:35.546 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:35.546 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:35.546 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:35.547 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:35.548 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:35.999 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-11-24T06:23:36.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-11-24T06:23:36.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:23:36.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:23:36.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:23:36.284 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:23:36.284 INFO:tasks.ceph:role: osd.7 2023-11-24T06:23:36.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:23:36.284 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:23:36.361 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:23:36.362 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:23:36.362 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:23:36.799 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2023-11-24T06:23:36.801 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2023-11-24T06:23:36.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:23:36.977 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:23:37.015 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.119+0000 7f9c994a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.119+0000 7f9c994a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.119+0000 7f9c994a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.119+0000 7f9c994a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.119+0000 7f9c994a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:23:37.123 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:37.120+0000 7f9c994a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:23:39.992 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:23:40.024 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:40.114 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:23:40.114 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:23:40.115 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:23:40.115 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:23:40.115 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:23:40.115 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:40.111+0000 7f68375af640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:23:42.977 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:23:43.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:43.102 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.098+0000 7f478e877640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:23:43.102 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.099+0000 7f478e877640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:23:43.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.099+0000 7f478e877640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:23:43.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.099+0000 7f478e877640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:23:43.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.099+0000 7f478e877640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:23:43.103 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:43.099+0000 7f478e877640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:23:44.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:44.818 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:23:45.971 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:23:46.002 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:46.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.093+0000 7fad9a104640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:23:46.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.093+0000 7fad9a104640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:23:46.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.094+0000 7fad9a104640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:23:46.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.094+0000 7fad9a104640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:23:46.097 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.094+0000 7fad9a104640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:23:46.098 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:46.094+0000 7fad9a104640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:23:48.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:23:48.997 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:23:48.997 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:48.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:23:49.029 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.029 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:23:49.100 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:23:49.169 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:23:49.237 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:23:49.306 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:23:49.375 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:23:49.443 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.443 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:23:49.475 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.475 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:23:49.545 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:23:49.615 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:23:49.684 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.684 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:23:49.753 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.753 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:23:49.821 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:23:49.888 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.888 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:23:49.907 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:49.907 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:23:49.967 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.967 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:23:49.984 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:49.984 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:23:50.041 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:23:50.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:23:50.043 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.y] 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBlcY5dGBAAZ7ISSGzT/OCSkRVGTBwkvA== 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBloXHaIBAA40BWWVcYcVbX+kM3wd4srw== 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBllY8dKBAAtxCS400KLv4rt6WIzc3a2g== 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2023-11-24T06:23:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBZQWBlQ7/gFxAA603E5V+l/7dkugKMUNQV3A== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBcQWBlnHtEFxAAr9h0RnKScdwcnLc2O9Pp4A== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBfQWBlmYVaGBAARyCei1YbVxxzy00k0W/Z6w== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBjQWBlVqdEABAAyD3s0xW8jXecRwdgtq8AwQ== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBlT3B0HRAANmHJk8Er7FHsAeQJuAnUKg== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBlRJM0LRAAToQLwjepMjKSY6cakPeFJw== 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2023-11-24T06:23:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBlwqqDNBAARSrqgGNEALaFrG0ZFStYgg== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBpQWBlyLk4BxAAauMGnn0akTG5zKYKcy9Vmw== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBsQWBlQjG4BhAAuv1Ja24nPcZ+SmnV8yuHgg== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBvQWBl4tb6BRAAcoydr7CybV8Ckv++5LFS+w== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQByQWBlIUmuBRAAO0eSyPnvmpUft7iB4vQ6dg== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[client.2] 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBWQWBlKtb6AhAAoaQhrX2DZgvNW0rcCB1iVg== 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi114.stdout:[client.1] 2023-11-24T06:23:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBWQWBlSFhWBhAA13v9kza/+snKVCmwnfK4eA== 2023-11-24T06:23:50.087 INFO:teuthology.orchestra.run.smithi114.stdout:[client.3] 2023-11-24T06:23:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBWQWBlwshBDBAAkMjk+usVBY7TYehxOhJdDw== 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBlcY5dGBAAZ7ISSGzT/OCSkRVGTBwkvA== 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBloXHaIBAA40BWWVcYcVbX+kM3wd4srw== 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2023-11-24T06:23:50.106 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBllY8dKBAAtxCS400KLv4rt6WIzc3a2g== 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBZQWBlQ7/gFxAA603E5V+l/7dkugKMUNQV3A== 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBcQWBlnHtEFxAAr9h0RnKScdwcnLc2O9Pp4A== 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBfQWBlmYVaGBAARyCei1YbVxxzy00k0W/Z6w== 2023-11-24T06:23:50.107 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBjQWBlVqdEABAAyD3s0xW8jXecRwdgtq8AwQ== 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBlT3B0HRAANmHJk8Er7FHsAeQJuAnUKg== 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBlRJM0LRAAToQLwjepMjKSY6cakPeFJw== 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2023-11-24T06:23:50.108 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBlwqqDNBAARSrqgGNEALaFrG0ZFStYgg== 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBpQWBlyLk4BxAAauMGnn0akTG5zKYKcy9Vmw== 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBsQWBlQjG4BhAAuv1Ja24nPcZ+SmnV8yuHgg== 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBvQWBl4tb6BRAAcoydr7CybV8Ckv++5LFS+w== 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2023-11-24T06:23:50.109 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQByQWBlIUmuBRAAO0eSyPnvmpUft7iB4vQ6dg== 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g== 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBWQWBlKtb6AhAAoaQhrX2DZgvNW0rcCB1iVg== 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBWQWBlSFhWBhAA13v9kza/+snKVCmwnfK4eA== 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2023-11-24T06:23:50.110 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQBWQWBlwshBDBAAkMjk+usVBY7TYehxOhJdDw== 2023-11-24T06:23:50.112 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:23:50.131 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:23:50.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.264 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.324 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.379 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:23:50.547 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:23:50.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.602 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.659 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.662 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:23:50.719 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.722 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.892 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:23:50.946 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:50.948 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.006 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.124 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:23:51.180 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:23:51.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:23:51.206 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:23:51.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:23:51.330 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:23:51.357 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:23:51.455 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:23:51.482 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:23:51.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:23:51.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:23:51.674 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:51.677 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:23:51.731 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:23:51.732 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:23:51.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:23:51.734 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:23:51.734 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:23:51.734 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:23:51.774 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:23:51.775 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:23:51.775 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:23:51.776 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:23:51.777 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:23:51.777 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:23:51.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:23:51.779 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:23:51.779 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:23:51.779 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:23:51.781 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:23:51.781 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:23:51.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:23:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.899+0000 7f719fdb0700 1 Processor -- start 2023-11-24T06:23:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 -- start start 2023-11-24T06:23:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719806b080 con 0x7f719815f9e0 2023-11-24T06:23:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7198074960 con 0x7f7198073f90 2023-11-24T06:23:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719fdb0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719815c290 con 0x7f71980cf610 2023-11-24T06:23:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.900+0000 7f719e34d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37052/0 (socket says 172.21.15.114:37052) 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 learned_addr learned my addr 172.21.15.114:0/449465428 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 msgr2=0x7f7198074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 msgr2=0x7f7198074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:23:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719db4c700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f71980add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:23:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719db4c700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f71980add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:23:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:23:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:51.901+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:23:52.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 msgr2=0x7f7198074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 msgr2=0x7f7198074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719db4c700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f71980add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719db4c700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f71980add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:23:52.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.102+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:23:52.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.107+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890653208 0 0) 0x7f719806b080 con 0x7f719815f9e0 2023-11-24T06:23:52.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71840035b0 con 0x7f719815f9e0 2023-11-24T06:23:52.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 774343163 0 0) 0x7f71840035b0 con 0x7f719815f9e0 2023-11-24T06:23:52.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f71980add40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:23:52.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f71980add40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 msgr2=0x7f7198074420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:23:52.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 --2- 172.21.15.114:0/449465428 >> v2:172.21.15.114:3300/0 conn(0x7f7198073f90 0x7f7198074420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980cf9f0 con 0x7f719815f9e0 2023-11-24T06:23:52.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f7188001670 con 0x7f719815f9e0 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 >> v1:172.21.15.114:6789/0 conn(0x7f719815f9e0 legacy=0x7f71980b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f7184003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004030 0x7f7184004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7184004d90 0x7f7184005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7198074960 con 0x7f7184004d90 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719815c290 con 0x7f7184004030 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.108+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718400bb10 con 0x7f71980cf610 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7184004d90 0x7f7184005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719db4c700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f7184003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 msgr2=0x7f7184003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:52.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71980cf610 0x7f7184003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004030 msgr2=0x7f7184004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004030 0x7f7184004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71840035b0 con 0x7f7184004d90 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719e34d700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7184004d90 0x7f7184005210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7194009940 tx=0x7f7184005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e36fde41fd31b43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194015670 con 0x7f7184004d90 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400c050 con 0x7f7184004d90 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.109+0000 7f719cb4a700 1 -- 172.21.15.114:0/449465428 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940159d0 con 0x7f7184004d90 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 -- 172.21.15.114:0/449465428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7184004d90 msgr2=0x7f7184005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 --2- 172.21.15.114:0/449465428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7184004d90 0x7f7184005210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7194009940 tx=0x7f7184005880 comp rx=0 tx=0).stop 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719e34d700 1 -- 172.21.15.114:0/449465428 reap_dead start 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 -- 172.21.15.114:0/449465428 shutdown_connections 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 -- 172.21.15.114:0/449465428 >> 172.21.15.114:0/449465428 conn(0x7f71980d0bf0 msgr2=0x7f71980d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:52.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 -- 172.21.15.114:0/449465428 shutdown_connections 2023-11-24T06:23:52.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.110+0000 7f719fdb0700 1 -- 172.21.15.114:0/449465428 wait complete. 2023-11-24T06:23:52.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.112+0000 7f719fdb0700 1 Processor -- start 2023-11-24T06:23:52.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.112+0000 7f719fdb0700 1 -- start start 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 0x7f7198155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71981561e0 0x7f7198151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71981566a0 con 0x7f719814fd50 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7198156980 con 0x7f7184004d90 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719fdb0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7198156c60 con 0x7f71981561e0 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719db4c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719db4c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:60974/0 (socket says 172.21.15.114:60974) 2023-11-24T06:23:52.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.113+0000 7f719db4c700 1 -- 172.21.15.114:0/3782667675 learned_addr learned my addr 172.21.15.114:0/3782667675 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 0x7f7198155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 -- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71981561e0 msgr2=0x7f7198151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71981561e0 0x7f7198151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 -- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 msgr2=0x7f71981427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 -- 172.21.15.114:0/3782667675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198156f40 con 0x7f719814fd50 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719db4c700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f719d34b700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 0x7f7198155ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f718800c730 tx=0x7f718800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.114+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188005480 con 0x7f719814fd50 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7188003680 con 0x7f719814fd50 2023-11-24T06:23:52.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7198157220 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188003a50 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7198151680 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f718801c440 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.115+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7188012740 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.116+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7198068fb0 con 0x7f719814fd50 2023-11-24T06:23:52.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.118+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7188021060 con 0x7f719814fd50 2023-11-24T06:23:52.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.194+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f719806c240 con 0x7f719814fd50 2023-11-24T06:23:52.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.196+0000 7f718e7fc700 1 -- 172.21.15.114:0/3782667675 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7188017070 con 0x7f719814fd50 2023-11-24T06:23:52.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.196+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 msgr2=0x7f7198155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.196+0000 7f719fdb0700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 0x7f7198155ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f718800c730 tx=0x7f718800ca40 comp rx=0 tx=0).stop 2023-11-24T06:23:52.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 shutdown_connections 2023-11-24T06:23:52.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71981561e0 0x7f7198151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7184004d90 0x7f71981427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 --2- 172.21.15.114:0/3782667675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f719814fd50 0x7f7198155ca0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 >> 172.21.15.114:0/3782667675 conn(0x7f71980d0bf0 msgr2=0x7f719806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:52.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 shutdown_connections 2023-11-24T06:23:52.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:52.197+0000 7f719fdb0700 1 -- 172.21.15.114:0/3782667675 wait complete. 2023-11-24T06:23:52.202 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2023-11-24T06:23:52.212 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:23:52.213 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:23:52.213 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:52.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:23:52.229 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:52.227+0000 7f7816bff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:23:52.231 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.228+0000 7f610415d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:23:52.239 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:52.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:23:52.294 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:52.292+0000 7f7816bff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:23:52.299 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.296+0000 7f610415d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:23:52.307 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:52.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:23:52.371 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:52.369+0000 7f7816bff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:23:52.373 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:23:52.373 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:23:52.380 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.377+0000 7f610415d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:23:52.405 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:52.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:23:52.432 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:52.432 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:23:52.452 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:52.450+0000 7f7816bff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:23:52.463 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.460+0000 7f610415d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:23:52.478 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:52.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:23:52.545 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:23:52.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:23:52.602 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:52.600+0000 7f7816bff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:23:52.605 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.602+0000 7f610415d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:23:52.611 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new dc893cbd-20b7-40ac-ae45-33325e2aaf47 0 2023-11-24T06:23:52.772 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.770+0000 7f4663f13700 1 Processor -- start 2023-11-24T06:23:52.773 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.770+0000 7f4663f13700 1 -- start start 2023-11-24T06:23:52.775 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f465c15e160 0x7f465c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.775 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.775 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f465c0744d0 0x7f465c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0b4500 con 0x7f465c15e160 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c074960 con 0x7f465c0744d0 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4663f13700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0b3c60 con 0x7f465c0afc90 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f46624b0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f465c0744d0 0x7f465c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f46614ae700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f46624b0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f465c0744d0 0x7f465c06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56178/0 (socket says 172.21.15.172:56178) 2023-11-24T06:23:52.776 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f46624b0700 1 -- 172.21.15.172:0/2244028052 learned_addr learned my addr 172.21.15.172:0/2244028052 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:52.777 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f46614ae700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54316/0 (socket says 172.21.15.172:54316) 2023-11-24T06:23:52.777 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.771+0000 7f4661caf700 1 --2- 172.21.15.172:0/2244028052 >> v2:172.21.15.114:3300/0 conn(0x7f465c15e160 0x7f465c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.777 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 -- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 msgr2=0x7f465c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:52.777 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.777 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 -- 172.21.15.172:0/2244028052 >> v2:172.21.15.114:3300/0 conn(0x7f465c15e160 msgr2=0x7f465c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:52.778 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 --2- 172.21.15.172:0/2244028052 >> v2:172.21.15.114:3300/0 conn(0x7f465c15e160 0x7f465c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.778 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 -- 172.21.15.172:0/2244028052 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0b58c0 con 0x7f465c0744d0 2023-11-24T06:23:52.778 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46614ae700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:52.778 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f46624b0700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c0744d0 0x7f465c06d930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f465c0d1cd0 tx=0x7f465400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b97be2720477f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:52.778 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f4660cad700 1 -- 172.21.15.172:0/2244028052 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465400e070 con 0x7f465c0744d0 2023-11-24T06:23:52.779 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f4660cad700 1 -- 172.21.15.172:0/2244028052 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465400b910 con 0x7f465c0744d0 2023-11-24T06:23:52.779 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f4660cad700 1 -- 172.21.15.172:0/2244028052 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465400bcc0 con 0x7f465c0744d0 2023-11-24T06:23:52.779 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f4663f13700 1 -- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c0744d0 msgr2=0x7f465c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.779 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.772+0000 7f4663f13700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c0744d0 0x7f465c06d930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f465c0d1cd0 tx=0x7f465400b330 comp rx=0 tx=0).stop 2023-11-24T06:23:52.779 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 -- 172.21.15.172:0/2244028052 shutdown_connections 2023-11-24T06:23:52.780 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c0744d0 0x7f465c06d930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.780 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 --2- 172.21.15.172:0/2244028052 >> v2:172.21.15.114:3300/0 conn(0x7f465c15e160 0x7f465c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.780 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 --2- 172.21.15.172:0/2244028052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.780 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 -- 172.21.15.172:0/2244028052 >> 172.21.15.172:0/2244028052 conn(0x7f465c0d0bf0 msgr2=0x7f465c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:52.780 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 -- 172.21.15.172:0/2244028052 shutdown_connections 2023-11-24T06:23:52.781 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.773+0000 7f4663f13700 1 -- 172.21.15.172:0/2244028052 wait complete. 2023-11-24T06:23:52.783 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.774+0000 7f4663f13700 1 Processor -- start 2023-11-24T06:23:52.783 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.774+0000 7f4663f13700 1 -- start start 2023-11-24T06:23:52.784 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f465c0744d0 0x7f465c145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.784 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.784 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 0x7f465c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:52.784 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1568d0 con 0x7f465c0744d0 2023-11-24T06:23:52.784 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c155cc0 con 0x7f465c15e160 2023-11-24T06:23:52.785 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c155fa0 con 0x7f465c0afc90 2023-11-24T06:23:52.785 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46614ae700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.785 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 0x7f465c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.785 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46614ae700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54322/0 (socket says 172.21.15.172:54322) 2023-11-24T06:23:52.785 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46614ae700 1 -- 172.21.15.172:0/1578276213 learned_addr learned my addr 172.21.15.172:0/1578276213 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4661caf700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f465c0744d0 0x7f465c145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 -- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 msgr2=0x7f465c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 -- 172.21.15.172:0/1578276213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f465c0744d0 msgr2=0x7f465c145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f465c0744d0 0x7f465c145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 -- 172.21.15.172:0/1578276213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0094a0 con 0x7f465c15e160 2023-11-24T06:23:52.787 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46614ae700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:52.787 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46624b0700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 0x7f465c156d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4654015000 tx=0x7f4654004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:52.787 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540092e0 con 0x7f465c15e160 2023-11-24T06:23:52.787 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465400ce80 con 0x7f465c15e160 2023-11-24T06:23:52.787 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4654009b30 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540044b0 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c156560 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.775+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f465401c440 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.776+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4654025750 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.776+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c1566f0 con 0x7f465c15e160 2023-11-24T06:23:52.788 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.778+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4654020030 con 0x7f465c15e160 2023-11-24T06:23:52.865 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.863+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dc893cbd-20b7-40ac-ae45-33325e2aaf47", "id": 0} v 0) v1 -- 0x7f465c068fb0 con 0x7f465c15e160 2023-11-24T06:23:52.868 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.866+0000 7f46527fc700 1 -- 172.21.15.172:0/1578276213 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc893cbd-20b7-40ac-ae45-33325e2aaf47", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4654032020 con 0x7f465c15e160 2023-11-24T06:23:52.868 INFO:teuthology.orchestra.run.smithi172.stdout:0 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 msgr2=0x7f465c156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 0x7f465c156d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4654015000 tx=0x7f4654004ad0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 shutdown_connections 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f465c15e160 0x7f465c156d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f465c0744d0 0x7f465c145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 --2- 172.21.15.172:0/1578276213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f465c0afc90 0x7f465c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 >> 172.21.15.172:0/1578276213 conn(0x7f465c0d0bf0 msgr2=0x7f465c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:52.870 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 shutdown_connections 2023-11-24T06:23:52.871 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:52.867+0000 7f4663f13700 1 -- 172.21.15.172:0/1578276213 wait complete. 2023-11-24T06:23:52.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new ce10f334-6cb1-4d9b-9cea-d54523268a95 1 2023-11-24T06:23:52.993 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:52.989+0000 7f610415d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:23:53.002 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.000+0000 7f15ee9d2700 1 Processor -- start 2023-11-24T06:23:53.002 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.000+0000 7f15ee9d2700 1 -- start start 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ee9d2700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15e815b370 0x7f15e815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ee9d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ee9d2700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e80ae5e0 con 0x7f15e81587f0 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ee9d2700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e80b4630 con 0x7f15e815b370 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ee9d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815be30 con 0x7f15e80b3c60 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ecf6f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15e815b370 0x7f15e815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.005 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ecf6f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54330/0 (socket says 172.21.15.172:54330) 2023-11-24T06:23:53.006 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15e815b370 0x7f15e815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56188/0 (socket says 172.21.15.172:56188) 2023-11-24T06:23:53.006 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e7fff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f15e81587f0 0x7f15e806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44932/0 (socket says 172.21.15.172:44932) 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ecf6f700 1 -- 172.21.15.172:0/1660262798 learned_addr learned my addr 172.21.15.172:0/1660262798 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 -- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 msgr2=0x7f15e80b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 -- 172.21.15.172:0/1660262798 >> v1:172.21.15.172:6789/0 conn(0x7f15e81587f0 legacy=0x7f15e806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 -- 172.21.15.172:0/1660262798 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e815c110 con 0x7f15e815b370 2023-11-24T06:23:53.007 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15ecf6f700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e77fe700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e815b370 0x7f15e815b800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00dc90 tx=0x7f15dc00f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f8bd4ad2c40a4aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e6ffd700 1 -- 172.21.15.172:0/1660262798 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc012070 con 0x7f15e815b370 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e6ffd700 1 -- 172.21.15.172:0/1660262798 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc00d2e0 con 0x7f15e815b370 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.001+0000 7f15e6ffd700 1 -- 172.21.15.172:0/1660262798 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00f8b0 con 0x7f15e815b370 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 -- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e815b370 msgr2=0x7f15e815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e815b370 0x7f15e815b800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00dc90 tx=0x7f15dc00f050 comp rx=0 tx=0).stop 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 -- 172.21.15.172:0/1660262798 shutdown_connections 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e80b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/1660262798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e815b370 0x7f15e815b800 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 -- 172.21.15.172:0/1660262798 >> 172.21.15.172:0/1660262798 conn(0x7f15e80d0bf0 msgr2=0x7f15e80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 -- 172.21.15.172:0/1660262798 shutdown_connections 2023-11-24T06:23:53.008 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.002+0000 7f15ee9d2700 1 -- 172.21.15.172:0/1660262798 wait complete. 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.003+0000 7f15ee9d2700 1 Processor -- start 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.003+0000 7f15ee9d2700 1 -- start start 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e815b370 0x7f15e8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.011 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81599d0 con 0x7f15e815b370 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815ae30 con 0x7f15e81587f0 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ee9d2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8158d50 con 0x7f15e80b3c60 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15e77fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15e77fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56194/0 (socket says 172.21.15.172:56194) 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54332/0 (socket says 172.21.15.172:54332) 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15e77fe700 1 -- 172.21.15.172:0/3886767222 learned_addr learned my addr 172.21.15.172:0/3886767222 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.004+0000 7f15ecf6f700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e815b370 0x7f15e8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 -- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 msgr2=0x7f15e815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.012 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 -- 172.21.15.172:0/3886767222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e815b370 msgr2=0x7f15e8152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e815b370 0x7f15e8152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 -- 172.21.15.172:0/3886767222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e000bb50 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e77fe700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e7fff700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c450 tx=0x7f15d800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8005cb0 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d8005e10 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15dc00d940 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80072d0 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8159310 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f15d801c440 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.005+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f15d80127b0 con 0x7f15e80b3c60 2023-11-24T06:23:53.013 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.006+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e8068fb0 con 0x7f15e80b3c60 2023-11-24T06:23:53.014 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:53.006+0000 7f7816bff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:23:53.014 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.008+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f15d8021060 con 0x7f15e80b3c60 2023-11-24T06:23:53.096 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.094+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce10f334-6cb1-4d9b-9cea-d54523268a95", "id": 1} v 0) v1 -- 0x7f15e815b8d0 con 0x7f15e80b3c60 2023-11-24T06:23:53.104 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.102+0000 7f15e4ff9700 1 -- 172.21.15.172:0/3886767222 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce10f334-6cb1-4d9b-9cea-d54523268a95", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f15d8017070 con 0x7f15e80b3c60 2023-11-24T06:23:53.104 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 msgr2=0x7f15e815a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c450 tx=0x7f15d800c760 comp rx=0 tx=0).stop 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 shutdown_connections 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15e815b370 0x7f15e8152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f15e80b3c60 0x7f15e815a8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 --2- 172.21.15.172:0/3886767222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15e81587f0 0x7f15e815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 >> 172.21.15.172:0/3886767222 conn(0x7f15e80d0bf0 msgr2=0x7f15e8155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 shutdown_connections 2023-11-24T06:23:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.103+0000 7f15ee9d2700 1 -- 172.21.15.172:0/3886767222 wait complete. 2023-11-24T06:23:53.114 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:53.111+0000 7f610415d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:23:53.118 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 7823a8b2-097f-44dc-9ddb-0f979186b3d7 2 2023-11-24T06:23:53.132 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:53.129+0000 7f7816bff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:23:53.196 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:53.192+0000 7f610415d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:23:53.220 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:53.217+0000 7f7816bff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:23:53.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.237+0000 7fd56fe9d700 1 Processor -- start 2023-11-24T06:23:53.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.237+0000 7fd56fe9d700 1 -- start start 2023-11-24T06:23:53.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56fe9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5680cc540 0x7fd56806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56fe9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 0x7fd5680b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56fe9d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5680b2f20 con 0x7fd56815a620 2023-11-24T06:23:53.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56fe9d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56806ef90 con 0x7fd5680cc540 2023-11-24T06:23:53.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56fe9d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56815aa00 con 0x7fd56806eb20 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56e43a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 0x7fd5680b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5680cc540 0x7fd56806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56e43a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 0x7fd5680b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54342/0 (socket says 172.21.15.172:54342) 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5680cc540 0x7fd56806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56200/0 (socket says 172.21.15.172:56200) 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56e43a700 1 -- 172.21.15.172:0/3550207293 learned_addr learned my addr 172.21.15.172:0/3550207293 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 -- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 msgr2=0x7fd5680b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 --2- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 0x7fd5680b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 -- 172.21.15.172:0/3550207293 >> v1:172.21.15.172:6789/0 conn(0x7fd56815a620 legacy=0x7fd5680ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56cc37700 1 -- 172.21.15.172:0/3550207293 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067180559 0 0) 0x7fd5680b2f20 con 0x7fd56815a620 2023-11-24T06:23:53.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 -- 172.21.15.172:0/3550207293 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680ce960 con 0x7fd5680cc540 2023-11-24T06:23:53.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.239+0000 7fd56d438700 1 --2- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5680cc540 0x7fd56806e5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd558009840 tx=0x7fd558009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42efe3941acdd00a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.240+0000 7fd56cc37700 1 -- 172.21.15.172:0/3550207293 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558011830 con 0x7fd5680cc540 2023-11-24T06:23:53.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.240+0000 7fd56cc37700 1 -- 172.21.15.172:0/3550207293 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd558011990 con 0x7fd5680cc540 2023-11-24T06:23:53.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.240+0000 7fd56cc37700 1 -- 172.21.15.172:0/3550207293 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558011d20 con 0x7fd5680cc540 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.240+0000 7fd56fe9d700 1 -- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5680cc540 msgr2=0x7fd56806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.240+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5680cc540 0x7fd56806e5e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd558009840 tx=0x7fd558009b50 comp rx=0 tx=0).stop 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 -- 172.21.15.172:0/3550207293 shutdown_connections 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56806eb20 0x7fd5680b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/3550207293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5680cc540 0x7fd56806e5e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 -- 172.21.15.172:0/3550207293 >> 172.21.15.172:0/3550207293 conn(0x7fd5680b2050 msgr2=0x7fd5680b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 -- 172.21.15.172:0/3550207293 shutdown_connections 2023-11-24T06:23:53.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.241+0000 7fd56fe9d700 1 -- 172.21.15.172:0/3550207293 wait complete. 2023-11-24T06:23:53.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.242+0000 7fd56fe9d700 1 Processor -- start 2023-11-24T06:23:53.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.243+0000 7fd56fe9d700 1 -- start start 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 0x7fd568153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56815a620 0x7fd56814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd568154a10 con 0x7fd5680cc540 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd568152f70 con 0x7fd56806eb20 2023-11-24T06:23:53.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56fe9d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd568153250 con 0x7fd56815a620 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56204/0 (socket says 172.21.15.172:56204) 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 -- 172.21.15.172:0/2474635397 learned_addr learned my addr 172.21.15.172:0/2474635397 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56e43a700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56815a620 0x7fd56814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56d438700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 0x7fd568153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 -- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56815a620 msgr2=0x7fd56814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56815a620 0x7fd56814ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 -- 172.21.15.172:0/2474635397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 msgr2=0x7fd568153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 0x7fd568153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 -- 172.21.15.172:0/2474635397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580094a0 con 0x7fd56806eb20 2023-11-24T06:23:53.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56d438700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 0x7fd568153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd56dc39700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd5640011e0 tx=0x7fd56400ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.244+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640054e0 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564003680 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564003a30 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd568153530 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56815d020 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd56401c440 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd564012820 con 0x7fd56806eb20 2023-11-24T06:23:53.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.245+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd568068d90 con 0x7fd56806eb20 2023-11-24T06:23:53.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.248+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd564017070 con 0x7fd56806eb20 2023-11-24T06:23:53.276 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:53.273+0000 7f610415d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:23:53.308 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:53.305+0000 7f7816bff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:23:53.335 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.332+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7823a8b2-097f-44dc-9ddb-0f979186b3d7", "id": 2} v 0) v1 -- 0x7fd56815d210 con 0x7fd56806eb20 2023-11-24T06:23:53.340 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.338+0000 7fd55e7fc700 1 -- 172.21.15.172:0/2474635397 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7823a8b2-097f-44dc-9ddb-0f979186b3d7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd56815d210 con 0x7fd56806eb20 2023-11-24T06:23:53.340 INFO:teuthology.orchestra.run.smithi172.stdout:2 2023-11-24T06:23:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 msgr2=0x7fd56815cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd5640011e0 tx=0x7fd56400ca30 comp rx=0 tx=0).stop 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 shutdown_connections 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd56815a620 0x7fd56814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd56806eb20 0x7fd56815cae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 --2- 172.21.15.172:0/2474635397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5680cc540 0x7fd568153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 >> 172.21.15.172:0/2474635397 conn(0x7fd5680b2050 msgr2=0x7fd56806dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 shutdown_connections 2023-11-24T06:23:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.339+0000 7fd56fe9d700 1 -- 172.21.15.172:0/2474635397 wait complete. 2023-11-24T06:23:53.353 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 140a162c-f0b9-4d80-9c96-08b6a2f01f43 3 2023-11-24T06:23:53.518 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.516+0000 7f60c71c4700 1 Processor -- start 2023-11-24T06:23:53.519 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.516+0000 7f60c71c4700 1 -- start start 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c71c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 0x7f60c00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c71c4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 0x7f60c0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c71c4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c00b3c60 con 0x7f60c00744d0 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c71c4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c00b4500 con 0x7f60c00afc90 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c71c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0074960 con 0x7f60c015e160 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c4f60700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 0x7f60c00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c5761700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f60c00744d0 0x7f60c006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44960/0 (socket says 172.21.15.172:44960) 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c5761700 1 -- 172.21.15.172:0/792424165 learned_addr learned my addr 172.21.15.172:0/792424165 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-11-24T06:23:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60bffff700 1 --2- 172.21.15.172:0/792424165 >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 0x7f60c0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c4f60700 1 -- 172.21.15.172:0/792424165 >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 msgr2=0x7f60c0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c4f60700 1 --2- 172.21.15.172:0/792424165 >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 0x7f60c0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c4f60700 1 -- 172.21.15.172:0/792424165 >> v1:172.21.15.172:6789/0 conn(0x7f60c00744d0 legacy=0x7f60c006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60bf7fe700 1 -- 172.21.15.172:0/792424165 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514893514 0 0) 0x7f60c00b3c60 con 0x7f60c00744d0 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60c4f60700 1 -- 172.21.15.172:0/792424165 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00b58c0 con 0x7f60c015e160 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.517+0000 7f60bffff700 1 --2- 172.21.15.172:0/792424165 >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 0x7f60c0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c4f60700 1 --2- 172.21.15.172:0/792424165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 0x7f60c00b5da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009840 tx=0x7f60b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a8abfecb07e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60bf7fe700 1 -- 172.21.15.172:0/792424165 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4011830 con 0x7f60c015e160 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60bf7fe700 1 -- 172.21.15.172:0/792424165 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b4011990 con 0x7f60c015e160 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60bf7fe700 1 -- 172.21.15.172:0/792424165 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4011d20 con 0x7f60c015e160 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 -- 172.21.15.172:0/792424165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 msgr2=0x7f60c00b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 --2- 172.21.15.172:0/792424165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 0x7f60c00b5da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009840 tx=0x7f60b4009b50 comp rx=0 tx=0).stop 2023-11-24T06:23:53.522 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 -- 172.21.15.172:0/792424165 shutdown_connections 2023-11-24T06:23:53.523 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 --2- 172.21.15.172:0/792424165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c015e160 0x7f60c00b5da0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.523 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 --2- 172.21.15.172:0/792424165 >> v2:172.21.15.114:3300/0 conn(0x7f60c00afc90 0x7f60c0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.523 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 -- 172.21.15.172:0/792424165 >> 172.21.15.172:0/792424165 conn(0x7f60c00d0bf0 msgr2=0x7f60c00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.523 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 -- 172.21.15.172:0/792424165 shutdown_connections 2023-11-24T06:23:53.523 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.518+0000 7f60c71c4700 1 -- 172.21.15.172:0/792424165 wait complete. 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 Processor -- start 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 -- start start 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c00afc90 0x7f60c00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c015e160 0x7f60c014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c014ea00 con 0x7f60c00afc90 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c5bd0 con 0x7f60c015e160 2023-11-24T06:23:53.525 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.520+0000 7f60c71c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c6750 con 0x7f60c00744d0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54356/0 (socket says 172.21.15.172:54356) 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 -- 172.21.15.172:0/1475505929 learned_addr learned my addr 172.21.15.172:0/1475505929 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c5761700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c015e160 0x7f60c014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bffff700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c00afc90 0x7f60c00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 -- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c015e160 msgr2=0x7f60c014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c015e160 0x7f60c014e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 -- 172.21.15.172:0/1475505929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c00afc90 msgr2=0x7f60c00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c00afc90 0x7f60c00c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 -- 172.21.15.172:0/1475505929 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40094a0 con 0x7f60c00744d0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c4f60700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009b90 tx=0x7f60b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.526 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40036a0 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b401f070 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c00c5e20 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4003a60 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c00c63e0 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f60b4026480 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.521+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f60b4021070 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.522+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0068fb0 con 0x7f60c00744d0 2023-11-24T06:23:53.527 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.524+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f60b402b020 con 0x7f60c00744d0 2023-11-24T06:23:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.608+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "140a162c-f0b9-4d80-9c96-08b6a2f01f43", "id": 3} v 0) v1 -- 0x7f60c006c140 con 0x7f60c00744d0 2023-11-24T06:23:53.614 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.611+0000 7f60bd7fa700 1 -- 172.21.15.172:0/1475505929 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "140a162c-f0b9-4d80-9c96-08b6a2f01f43", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f60b4015020 con 0x7f60c00744d0 2023-11-24T06:23:53.614 INFO:teuthology.orchestra.run.smithi172.stdout:3 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 msgr2=0x7f60c014b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009b90 tx=0x7f60b4004750 comp rx=0 tx=0).stop 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 shutdown_connections 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c015e160 0x7f60c014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00744d0 0x7f60c014b260 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 --2- 172.21.15.172:0/1475505929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60c00afc90 0x7f60c00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.612+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 >> 172.21.15.172:0/1475505929 conn(0x7f60c00d0bf0 msgr2=0x7f60c006e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.615 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.613+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 shutdown_connections 2023-11-24T06:23:53.616 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.613+0000 7f60c71c4700 1 -- 172.21.15.172:0/1475505929 wait complete. 2023-11-24T06:23:53.627 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 6bdec982-f0c6-4d00-b5e2-8d5cdceeb597 4 2023-11-24T06:23:53.717 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:53.714+0000 7f610415d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:23:53.748 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.746+0000 7f91475e3700 1 Processor -- start 2023-11-24T06:23:53.749 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.746+0000 7f91475e3700 1 -- start start 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f91475e3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f914015e160 0x7f91400b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f91475e3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f91475e3700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400b3c60 con 0x7f91400744d0 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f91475e3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400b4500 con 0x7f914015e160 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f91475e3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9140074960 con 0x7f91400afc90 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54362/0 (socket says 172.21.15.172:54362) 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 -- 172.21.15.172:0/1875938817 learned_addr learned my addr 172.21.15.172:0/1875938817 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.751 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f914537f700 1 --2- 172.21.15.172:0/1875938817 >> v2:172.21.15.114:3300/0 conn(0x7f914015e160 0x7f91400b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 -- 172.21.15.172:0/1875938817 >> v2:172.21.15.114:3300/0 conn(0x7f914015e160 msgr2=0x7f91400b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 --2- 172.21.15.172:0/1875938817 >> v2:172.21.15.114:3300/0 conn(0x7f914015e160 0x7f91400b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.747+0000 7f9144b7e700 1 -- 172.21.15.172:0/1875938817 >> v1:172.21.15.114:6789/0 conn(0x7f91400744d0 legacy=0x7f914006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f9144b7e700 1 -- 172.21.15.172:0/1875938817 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400b58c0 con 0x7f91400afc90 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f9144b7e700 1 --2- 172.21.15.172:0/1875938817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f913c0098f0 tx=0x7f913c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8adcd05422718e43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f9137fff700 1 -- 172.21.15.172:0/1875938817 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c011830 con 0x7f91400afc90 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f9137fff700 1 -- 172.21.15.172:0/1875938817 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c011990 con 0x7f91400afc90 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f9137fff700 1 -- 172.21.15.172:0/1875938817 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c011d20 con 0x7f91400afc90 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 -- 172.21.15.172:0/1875938817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 msgr2=0x7f9140073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.752 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 --2- 172.21.15.172:0/1875938817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f913c0098f0 tx=0x7f913c009c00 comp rx=0 tx=0).stop 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 -- 172.21.15.172:0/1875938817 shutdown_connections 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 --2- 172.21.15.172:0/1875938817 >> v2:172.21.15.114:3300/0 conn(0x7f914015e160 0x7f91400b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 --2- 172.21.15.172:0/1875938817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f9140073f90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.748+0000 7f91475e3700 1 -- 172.21.15.172:0/1875938817 >> 172.21.15.172:0/1875938817 conn(0x7f91400d0bf0 msgr2=0x7f91400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.749+0000 7f91475e3700 1 -- 172.21.15.172:0/1875938817 shutdown_connections 2023-11-24T06:23:53.753 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.749+0000 7f91475e3700 1 -- 172.21.15.172:0/1875938817 wait complete. 2023-11-24T06:23:53.755 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.750+0000 7f91475e3700 1 Processor -- start 2023-11-24T06:23:53.755 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.750+0000 7f91475e3700 1 -- start start 2023-11-24T06:23:53.755 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91400744d0 0x7f91400c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.755 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 0x7f91400c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c7660 con 0x7f91400744d0 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c8830 con 0x7f914015e160 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400cc220 con 0x7f91400afc90 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54370/0 (socket says 172.21.15.172:54370) 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 -- 172.21.15.172:0/4087465923 learned_addr learned my addr 172.21.15.172:0/4087465923 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9145b80700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 0x7f91400c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f914537f700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91400744d0 0x7f91400c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 -- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 msgr2=0x7f91400c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 0x7f91400c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.756 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 -- 172.21.15.172:0/4087465923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91400744d0 msgr2=0x7f91400c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91400744d0 0x7f91400c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 -- 172.21.15.172:0/4087465923 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009550 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9145b80700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 0x7f91400c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9144b7e700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f913c009c40 tx=0x7f913c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c004400 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c01f070 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400c8ac0 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.751+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0038e0 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.752+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91400c9300 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.752+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f913c021030 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.752+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f913c01adc0 con 0x7f91400afc90 2023-11-24T06:23:53.757 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.752+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914015d360 con 0x7f91400afc90 2023-11-24T06:23:53.758 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.755+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f913c015020 con 0x7f91400afc90 2023-11-24T06:23:53.769 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:53.766+0000 7f7816bff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:23:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.837+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6bdec982-f0c6-4d00-b5e2-8d5cdceeb597", "id": 4} v 0) v1 -- 0x7f9140068fb0 con 0x7f91400afc90 2023-11-24T06:23:53.845 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.842+0000 7f9135ffb700 1 -- 172.21.15.172:0/4087465923 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6bdec982-f0c6-4d00-b5e2-8d5cdceeb597", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f913c01a450 con 0x7f91400afc90 2023-11-24T06:23:53.846 INFO:teuthology.orchestra.run.smithi172.stdout:4 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.843+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 msgr2=0x7f91400c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.843+0000 7f91475e3700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f913c009c40 tx=0x7f913c004750 comp rx=0 tx=0).stop 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 shutdown_connections 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f914015e160 0x7f91400c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91400744d0 0x7f91400c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 --2- 172.21.15.172:0/4087465923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91400afc90 0x7f91400c82f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 >> 172.21.15.172:0/4087465923 conn(0x7f91400d0bf0 msgr2=0x7f9140071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 shutdown_connections 2023-11-24T06:23:53.847 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.844+0000 7f91475e3700 1 -- 172.21.15.172:0/4087465923 wait complete. 2023-11-24T06:23:53.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new fb61e375-0fe3-488a-9b10-fab492072d79 5 2023-11-24T06:23:53.980 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.978+0000 7fed5ff00700 1 Processor -- start 2023-11-24T06:23:53.980 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.978+0000 7fed5ff00700 1 -- start start 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5ff00700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed5815ba30 0x7fed5806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5ff00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5ff00700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5815f280 con 0x7fed5815e540 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5ff00700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5815f090 con 0x7fed5815ba30 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5ff00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5815f560 con 0x7fed58162800 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54376/0 (socket says 172.21.15.172:54376) 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5e49d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fed5815e540 0x7fed5815e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44984/0 (socket says 172.21.15.172:44984) 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 -- 172.21.15.172:0/1673236432 learned_addr learned my addr 172.21.15.172:0/1673236432 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5dc9c700 1 --2- 172.21.15.172:0/1673236432 >> v2:172.21.15.114:3300/0 conn(0x7fed5815ba30 0x7fed5806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5cc9a700 1 -- 172.21.15.172:0/1673236432 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853992156 0 0) 0x7fed5815f280 con 0x7fed5815e540 2023-11-24T06:23:53.983 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5cc9a700 1 -- 172.21.15.172:0/1673236432 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed440035b0 con 0x7fed5815e540 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 -- 172.21.15.172:0/1673236432 >> v2:172.21.15.114:3300/0 conn(0x7fed5815ba30 msgr2=0x7fed5806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 --2- 172.21.15.172:0/1673236432 >> v2:172.21.15.114:3300/0 conn(0x7fed5815ba30 0x7fed5806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 -- 172.21.15.172:0/1673236432 >> v1:172.21.15.172:6789/0 conn(0x7fed5815e540 legacy=0x7fed5815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 -- 172.21.15.172:0/1673236432 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58161d20 con 0x7fed58162800 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5d49b700 1 --2- 172.21.15.172:0/1673236432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed480098f0 tx=0x7fed48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b634e455bc2e065 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.979+0000 7fed5cc9a700 1 -- 172.21.15.172:0/1673236432 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011830 con 0x7fed58162800 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5cc9a700 1 -- 172.21.15.172:0/1673236432 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48011990 con 0x7fed58162800 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5cc9a700 1 -- 172.21.15.172:0/1673236432 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011d20 con 0x7fed58162800 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 -- 172.21.15.172:0/1673236432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 msgr2=0x7fed5806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 --2- 172.21.15.172:0/1673236432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed480098f0 tx=0x7fed48009c00 comp rx=0 tx=0).stop 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 -- 172.21.15.172:0/1673236432 shutdown_connections 2023-11-24T06:23:53.984 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 --2- 172.21.15.172:0/1673236432 >> v2:172.21.15.114:3300/0 conn(0x7fed5815ba30 0x7fed5806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.985 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 --2- 172.21.15.172:0/1673236432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed58162800 0x7fed5806c540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.985 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 -- 172.21.15.172:0/1673236432 >> 172.21.15.172:0/1673236432 conn(0x7fed58071a70 msgr2=0x7fed58071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:53.985 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 -- 172.21.15.172:0/1673236432 shutdown_connections 2023-11-24T06:23:53.985 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.980+0000 7fed5ff00700 1 -- 172.21.15.172:0/1673236432 wait complete. 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.981+0000 7fed5ff00700 1 Processor -- start 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 -- start start 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed58162800 0x7fed5814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:53.987 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5814c110 con 0x7fed58162800 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed581f3e30 con 0x7fed5815ba30 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5ff00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed581f4110 con 0x7fed5815e540 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5d49b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5dc9c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5d49b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54382/0 (socket says 172.21.15.172:54382) 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5dc9c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56240/0 (socket says 172.21.15.172:56240) 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5d49b700 1 -- 172.21.15.172:0/44217883 learned_addr learned my addr 172.21.15.172:0/44217883 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.982+0000 7fed5e49d700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed58162800 0x7fed5814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 -- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 msgr2=0x7fed58148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 -- 172.21.15.172:0/44217883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed58162800 msgr2=0x7fed5814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed58162800 0x7fed5814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:53.988 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 -- 172.21.15.172:0/44217883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48009550 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5d49b700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5dc9c700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e900 tx=0x7fed5400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54013680 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540137e0 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5815bf90 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54013b50 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed581f46d0 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fed54005a20 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.983+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fed540067a0 con 0x7fed5815ba30 2023-11-24T06:23:53.989 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.984+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed58068fb0 con 0x7fed5815ba30 2023-11-24T06:23:53.990 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:53.986+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fed54021060 con 0x7fed5815ba30 2023-11-24T06:23:54.073 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.071+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fb61e375-0fe3-488a-9b10-fab492072d79", "id": 5} v 0) v1 -- 0x7fed5806bd30 con 0x7fed5815ba30 2023-11-24T06:23:54.082 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.079+0000 7fed4e7fc700 1 -- 172.21.15.172:0/44217883 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb61e375-0fe3-488a-9b10-fab492072d79", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed54017070 con 0x7fed5815ba30 2023-11-24T06:23:54.082 INFO:teuthology.orchestra.run.smithi172.stdout:5 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 msgr2=0x7fed581f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e900 tx=0x7fed5400ec10 comp rx=0 tx=0).stop 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 shutdown_connections 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed58162800 0x7fed5814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed5815ba30 0x7fed581f37c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 --2- 172.21.15.172:0/44217883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed5815e540 0x7fed58148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 >> 172.21.15.172:0/44217883 conn(0x7fed58071a70 msgr2=0x7fed58071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.083 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 shutdown_connections 2023-11-24T06:23:54.084 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.080+0000 7fed5ff00700 1 -- 172.21.15.172:0/44217883 wait complete. 2023-11-24T06:23:54.094 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 4f66bf03-958c-4802-b95d-f883e98831cb 6 2023-11-24T06:23:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.238+0000 7f5fe4bdb700 1 Processor -- start 2023-11-24T06:23:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.238+0000 7f5fe4bdb700 1 -- start start 2023-11-24T06:23:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fe4bdb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5fe015e140 0x7f5fe00d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fe4bdb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fe00d39d0 0x7f5fe00d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fe4bdb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe00b80e0 con 0x7f5fe00b98c0 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fe4bdb700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe015e520 con 0x7f5fe015e140 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fe4bdb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe00d2930 con 0x7f5fe00d39d0 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fe00d39d0 0x7f5fe00d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fded9d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5fe00b98c0 0x7f5fe00b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44996/0 (socket says 172.21.15.172:44996) 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fe00d39d0 0x7f5fe00d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56246/0 (socket says 172.21.15.172:56246) 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fded9d700 1 -- 172.21.15.172:0/2364222959 learned_addr learned my addr 172.21.15.172:0/2364222959 (peer_addr_for_me v1:172.21.15.172:0/0) 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fde59c700 1 --2- 172.21.15.172:0/2364222959 >> v2:172.21.15.114:3300/0 conn(0x7f5fe015e140 0x7f5fe00d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 -- 172.21.15.172:0/2364222959 >> v2:172.21.15.114:3300/0 conn(0x7f5fe015e140 msgr2=0x7f5fe00d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 --2- 172.21.15.172:0/2364222959 >> v2:172.21.15.114:3300/0 conn(0x7f5fe015e140 0x7f5fe00d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 -- 172.21.15.172:0/2364222959 >> v1:172.21.15.172:6789/0 conn(0x7f5fe00b98c0 legacy=0x7f5fe00b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fdd59a700 1 -- 172.21.15.172:0/2364222959 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830184234 0 0) 0x7f5fd8001490 con 0x7f5fe00b98c0 2023-11-24T06:23:54.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 -- 172.21.15.172:0/2364222959 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00d2c10 con 0x7f5fe00d39d0 2023-11-24T06:23:54.244 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fddd9b700 1 --2- 172.21.15.172:0/2364222959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe00d39d0 0x7f5fe00d4400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00d4b20 tx=0x7f5fd0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dda8346f258ae64 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fdd59a700 1 -- 172.21.15.172:0/2364222959 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011830 con 0x7f5fe00d39d0 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.239+0000 7f5fdd59a700 1 -- 172.21.15.172:0/2364222959 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0011990 con 0x7f5fe00d39d0 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fdd59a700 1 -- 172.21.15.172:0/2364222959 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011d20 con 0x7f5fe00d39d0 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/2364222959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe00d39d0 msgr2=0x7f5fe00d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/2364222959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe00d39d0 0x7f5fe00d4400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00d4b20 tx=0x7f5fd0009b20 comp rx=0 tx=0).stop 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/2364222959 shutdown_connections 2023-11-24T06:23:54.245 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/2364222959 >> v2:172.21.15.114:3300/0 conn(0x7f5fe015e140 0x7f5fe00d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/2364222959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe00d39d0 0x7f5fe00d4400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/2364222959 >> 172.21.15.172:0/2364222959 conn(0x7f5fe006b410 msgr2=0x7f5fe006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/2364222959 shutdown_connections 2023-11-24T06:23:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.240+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/2364222959 wait complete. 2023-11-24T06:23:54.248 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 Processor -- start 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 -- start start 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe00d39d0 0x7f5fe01f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 0x7f5fe01f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00b64f0 con 0x7f5fe00d39d0 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe01f36c0 con 0x7f5fe015e140 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fe4bdb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00c8c80 con 0x7f5fe00b98c0 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fde59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54392/0 (socket says 172.21.15.172:54392) 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fded9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 0x7f5fe01f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.242+0000 7f5fde59c700 1 -- 172.21.15.172:0/878908937 learned_addr learned my addr 172.21.15.172:0/878908937 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:54.249 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fddd9b700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe00d39d0 0x7f5fe01f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 -- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 msgr2=0x7f5fe01f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 0x7f5fe01f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 -- 172.21.15.172:0/878908937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe00d39d0 msgr2=0x7f5fe01f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe00d39d0 0x7f5fe01f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 -- 172.21.15.172:0/878908937 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00094a0 con 0x7f5fe00b98c0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fded9d700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 0x7f5fe01f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fde59c700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e870 tx=0x7f5fd400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4009590 con 0x7f5fe00b98c0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4013680 con 0x7f5fe00b98c0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe00b5970 con 0x7f5fe00b98c0 2023-11-24T06:23:54.250 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40139f0 con 0x7f5fe00b98c0 2023-11-24T06:23:54.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe00b6210 con 0x7f5fe00b98c0 2023-11-24T06:23:54.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5fd4005a20 con 0x7f5fe00b98c0 2023-11-24T06:23:54.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.243+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5fd40066b0 con 0x7f5fe00b98c0 2023-11-24T06:23:54.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.244+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe0068fb0 con 0x7f5fe00b98c0 2023-11-24T06:23:54.251 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.246+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5fd4021060 con 0x7f5fe00b98c0 2023-11-24T06:23:54.333 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.331+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f66bf03-958c-4802-b95d-f883e98831cb", "id": 6} v 0) v1 -- 0x7f5fe006bb30 con 0x7f5fe00b98c0 2023-11-24T06:23:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.339+0000 7f5fceffd700 1 -- 172.21.15.172:0/878908937 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f66bf03-958c-4802-b95d-f883e98831cb", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5fd4017070 con 0x7f5fe00b98c0 2023-11-24T06:23:54.342 INFO:teuthology.orchestra.run.smithi172.stdout:6 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 msgr2=0x7f5fe0149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e870 tx=0x7f5fd400eb80 comp rx=0 tx=0).stop 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 shutdown_connections 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe015e140 0x7f5fe01f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fe00b98c0 0x7f5fe0149cb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 --2- 172.21.15.172:0/878908937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe00d39d0 0x7f5fe01f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 >> 172.21.15.172:0/878908937 conn(0x7f5fe006b410 msgr2=0x7f5fe00b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 shutdown_connections 2023-11-24T06:23:54.343 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.340+0000 7f5fe4bdb700 1 -- 172.21.15.172:0/878908937 wait complete. 2023-11-24T06:23:54.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new b1be8744-fdb3-40b7-825b-492a3375649d 7 2023-11-24T06:23:54.363 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:54.360+0000 7f610415d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:23:54.432 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:54.428+0000 7f610415d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:23:54.439 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:54.437+0000 7f7816bff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:23:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.483+0000 7fbd75c29700 1 Processor -- start 2023-11-24T06:23:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.483+0000 7fbd75c29700 1 -- start start 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd7015e160 0x7fbd700b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbd700744d0 0x7fbd7006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd700b4500 con 0x7fbd700744d0 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd70074960 con 0x7fbd7015e160 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd75c29700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700b3c60 con 0x7fbd700afc90 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd6effd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd6effd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54400/0 (socket says 172.21.15.172:54400) 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd6effd700 1 -- 172.21.15.172:0/3651934424 learned_addr learned my addr 172.21.15.172:0/3651934424 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.484+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/3651934424 >> v2:172.21.15.172:3300/0 conn(0x7fbd7015e160 0x7fbd700b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6ffff700 1 --2- 172.21.15.172:0/3651934424 >> v2:172.21.15.114:3300/0 conn(0x7fbd700744d0 0x7fbd7006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 msgr2=0x7fbd70073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/3651934424 >> v2:172.21.15.114:3300/0 conn(0x7fbd700744d0 msgr2=0x7fbd7006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/3651934424 >> v2:172.21.15.114:3300/0 conn(0x7fbd700744d0 0x7fbd7006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/3651934424 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700b58c0 con 0x7fbd7015e160 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6effd700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd7015e160 0x7fbd700b5da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd7009efa0 tx=0x7fbd60009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d365047fcc843654 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6e7fc700 1 -- 172.21.15.172:0/3651934424 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60011830 con 0x7fbd7015e160 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6e7fc700 1 -- 172.21.15.172:0/3651934424 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60011990 con 0x7fbd7015e160 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.485+0000 7fbd6e7fc700 1 -- 172.21.15.172:0/3651934424 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60011d20 con 0x7fbd7015e160 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 -- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd7015e160 msgr2=0x7fbd700b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd7015e160 0x7fbd700b5da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd7009efa0 tx=0x7fbd60009b20 comp rx=0 tx=0).stop 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 -- 172.21.15.172:0/3651934424 shutdown_connections 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 --2- 172.21.15.172:0/3651934424 >> v2:172.21.15.114:3300/0 conn(0x7fbd700744d0 0x7fbd7006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd7015e160 0x7fbd700b5da0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 --2- 172.21.15.172:0/3651934424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd70073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 -- 172.21.15.172:0/3651934424 >> 172.21.15.172:0/3651934424 conn(0x7fbd700d0bf0 msgr2=0x7fbd700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 -- 172.21.15.172:0/3651934424 shutdown_connections 2023-11-24T06:23:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.486+0000 7fbd75c29700 1 -- 172.21.15.172:0/3651934424 wait complete. 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.487+0000 7fbd75c29700 1 Processor -- start 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 -- start start 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 0x7fbd701f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd700c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd7015e160 0x7fbd7006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7015f5b0 con 0x7fbd7015e160 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701f4b80 con 0x7fbd700744d0 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.488+0000 7fbd75c29700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7015e930 con 0x7fbd700afc90 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6effd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd700c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 0x7fbd701f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6effd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd700c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:54406/0 (socket says 172.21.15.172:54406) 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6effd700 1 -- 172.21.15.172:0/1774122429 learned_addr learned my addr 172.21.15.172:0/1774122429 (peer_addr_for_me v2:172.21.15.172:0/0) 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6ffff700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd7015e160 0x7fbd7006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 msgr2=0x7fbd700c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd700c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/1774122429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd7015e160 msgr2=0x7fbd7006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd7015e160 0x7fbd7006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 -- 172.21.15.172:0/1774122429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009550 con 0x7fbd700744d0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd6f7fe700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 0x7fbd701f4640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd60016000 tx=0x7fbd60004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60004510 con 0x7fbd700744d0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6001e070 con 0x7fbd700744d0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd600094a0 con 0x7fbd700744d0 2023-11-24T06:23:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd600038e0 con 0x7fbd700744d0 2023-11-24T06:23:54.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7015eef0 con 0x7fbd700744d0 2023-11-24T06:23:54.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbd60003d40 con 0x7fbd700744d0 2023-11-24T06:23:54.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.489+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fbd6001aa80 con 0x7fbd700744d0 2023-11-24T06:23:54.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.490+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd70068fb0 con 0x7fbd700744d0 2023-11-24T06:23:54.496 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.492+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbd60020070 con 0x7fbd700744d0 2023-11-24T06:23:54.520 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:54.518+0000 7f7816bff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:23:54.578 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.576+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b1be8744-fdb3-40b7-825b-492a3375649d", "id": 7} v 0) v1 -- 0x7fbd7006c140 con 0x7fbd700744d0 2023-11-24T06:23:54.584 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.582+0000 7fbd57fff700 1 -- 172.21.15.172:0/1774122429 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1be8744-fdb3-40b7-825b-492a3375649d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbd60015020 con 0x7fbd700744d0 2023-11-24T06:23:54.585 INFO:teuthology.orchestra.run.smithi172.stdout:7 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 msgr2=0x7fbd701f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 0x7fbd701f4640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd60016000 tx=0x7fbd60004750 comp rx=0 tx=0).stop 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 shutdown_connections 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd7015e160 0x7fbd7006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd700744d0 0x7fbd701f4640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 --2- 172.21.15.172:0/1774122429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd700afc90 0x7fbd700c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 >> 172.21.15.172:0/1774122429 conn(0x7fbd700d0bf0 msgr2=0x7fbd7006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 shutdown_connections 2023-11-24T06:23:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2023-11-24T06:23:54.583+0000 7fbd75c29700 1 -- 172.21.15.172:0/1774122429 wait complete. 2023-11-24T06:23:54.598 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:23:54.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:23:54.601 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:23:54.601 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:23:54.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:23:54.603 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:23:54.603 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:23:54.603 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:23:54.605 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:23:54.605 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:23:54.605 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:23:54.608 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:23:54.608 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:23:54.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:23:54.610 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:23:54.610 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:23:54.610 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:23:54.612 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:23:54.612 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:23:54.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:23:54.614 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:23:54.614 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:23:54.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:23:54.616 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:23:54.617 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:23:54.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.760+0000 7f40aa573700 1 Processor -- start 2023-11-24T06:23:54.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.760+0000 7f40aa573700 1 -- start start 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40aa573700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40aa573700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40aa573700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40af8c0 con 0x7f40a40cfce0 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40aa573700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40b0790 con 0x7f40a40cf860 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40aa573700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40afb10 con 0x7f40a415afc0 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a37fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a8b10700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f40a40cfce0 0x7f40a40b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:46426/0 (socket says 172.21.15.114:46426) 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a37fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:32778/0 (socket says 172.21.15.114:32778) 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:59894/0 (socket says 172.21.15.114:59894) 2023-11-24T06:23:54.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a8b10700 1 -- 172.21.15.114:0/2967277379 learned_addr learned my addr 172.21.15.114:0/2967277379 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 -- 172.21.15.114:0/2967277379 >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 msgr2=0x7f40a40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 --2- 172.21.15.114:0/2967277379 >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 -- 172.21.15.114:0/2967277379 >> v1:172.21.15.172:6789/0 conn(0x7f40a40cfce0 legacy=0x7f40a40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a3fff700 1 -- 172.21.15.114:0/2967277379 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40afdf0 con 0x7f40a415afc0 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.761+0000 7f40a37fe700 1 --2- 172.21.15.114:0/2967277379 >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40a3fff700 1 --2- 172.21.15.114:0/2967277379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4094009840 tx=0x7f4094009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5ea2d6deb447f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40a2ffd700 1 -- 172.21.15.114:0/2967277379 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094011830 con 0x7f40a415afc0 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40a2ffd700 1 -- 172.21.15.114:0/2967277379 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094011990 con 0x7f40a415afc0 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40a2ffd700 1 -- 172.21.15.114:0/2967277379 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094011d20 con 0x7f40a415afc0 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 -- 172.21.15.114:0/2967277379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 msgr2=0x7f40a40b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 --2- 172.21.15.114:0/2967277379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4094009840 tx=0x7f4094009b50 comp rx=0 tx=0).stop 2023-11-24T06:23:54.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 -- 172.21.15.114:0/2967277379 shutdown_connections 2023-11-24T06:23:54.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 --2- 172.21.15.114:0/2967277379 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a415afc0 0x7f40a40b0bb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 --2- 172.21.15.114:0/2967277379 >> v2:172.21.15.172:3300/0 conn(0x7f40a40cf860 0x7f40a40b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 -- 172.21.15.114:0/2967277379 >> 172.21.15.114:0/2967277379 conn(0x7f40a40b57b0 msgr2=0x7f40a40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 -- 172.21.15.114:0/2967277379 shutdown_connections 2023-11-24T06:23:54.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.762+0000 7f40aa573700 1 -- 172.21.15.114:0/2967277379 wait complete. 2023-11-24T06:23:54.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.763+0000 7f40aa573700 1 Processor -- start 2023-11-24T06:23:54.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 -- start start 2023-11-24T06:23:54.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a40cfce0 0x7f40a40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40c9b50 con 0x7f40a415afc0 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40c9d80 con 0x7f40a40cf860 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.764+0000 7f40aa573700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40ca060 con 0x7f40a40cfce0 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a8b10700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:32782/0 (socket says 172.21.15.114:32782) 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a8b10700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37128/0 (socket says 172.21.15.114:37128) 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 -- 172.21.15.114:0/1432906086 learned_addr learned my addr 172.21.15.114:0/1432906086 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a37fe700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a40cfce0 0x7f40a40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 -- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a40cfce0 msgr2=0x7f40a40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a40cfce0 0x7f40a40c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 -- 172.21.15.114:0/1432906086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 msgr2=0x7f40a40cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 -- 172.21.15.114:0/1432906086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980094a0 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a8b10700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a3fff700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40940090b0 tx=0x7f4094004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940036a0 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409401e070 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40940094a0 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094003a60 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a40cac20 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f409402d450 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.765+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f409401aaf0 con 0x7f40a40cf860 2023-11-24T06:23:54.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.766+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4067f60 con 0x7f40a40cf860 2023-11-24T06:23:54.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.769+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4094020070 con 0x7f40a40cf860 2023-11-24T06:23:54.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.838+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40a40715d0 con 0x7f40a40cf860 2023-11-24T06:23:54.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.839+0000 7f40a0ff9700 1 -- 172.21.15.114:0/1432906086 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4094015020 con 0x7f40a40cf860 2023-11-24T06:23:54.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:23:54.844 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:23:54.578009+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:23:54.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 msgr2=0x7f40a415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:54.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40940090b0 tx=0x7f4094004750 comp rx=0 tx=0).stop 2023-11-24T06:23:54.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 shutdown_connections 2023-11-24T06:23:54.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40a415afc0 0x7f40a40cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40a40cf860 0x7f40a415deb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 --2- 172.21.15.114:0/1432906086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40a40cfce0 0x7f40a40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:54.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 >> 172.21.15.114:0/1432906086 conn(0x7f40a40b57b0 msgr2=0x7f40a40ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:54.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 shutdown_connections 2023-11-24T06:23:54.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:23:54.840+0000 7f40aa573700 1 -- 172.21.15.114:0/1432906086 wait complete. 2023-11-24T06:23:54.855 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:23:54.856 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:23:55.173 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.170+0000 7f610415d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:23:55.212 INFO:tasks.ceph.osd.1.smithi114.stderr:2023-11-24T06:23:55.208+0000 7fee8ce1e640 -1 Falling back to public interface 2023-11-24T06:23:55.226 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-11-24T06:23:55.223+0000 7f97c53e4640 -1 Falling back to public interface 2023-11-24T06:23:55.230 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-11-24T06:23:55.228+0000 7fdc9e901640 -1 Falling back to public interface 2023-11-24T06:23:55.233 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-11-24T06:23:55.231+0000 7fd4c4213640 -1 Falling back to public interface 2023-11-24T06:23:55.235 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-11-24T06:23:55.233+0000 7fc01fb02640 -1 Falling back to public interface 2023-11-24T06:23:55.250 INFO:tasks.ceph.osd.0.smithi114.stderr:2023-11-24T06:23:55.246+0000 7fc59a085640 -1 Falling back to public interface 2023-11-24T06:23:55.251 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.248+0000 7f610415d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:23:55.254 INFO:tasks.ceph.osd.2.smithi114.stderr:2023-11-24T06:23:55.251+0000 7faa8e786640 -1 Falling back to public interface 2023-11-24T06:23:55.254 INFO:tasks.ceph.osd.3.smithi114.stderr:2023-11-24T06:23:55.251+0000 7fe64a58c640 -1 Falling back to public interface 2023-11-24T06:23:55.260 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:55.258+0000 7f7816bff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:23:55.338 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:55.335+0000 7f7816bff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:23:55.397 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.394+0000 7f610415d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:23:55.479 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:55.477+0000 7f7816bff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:23:55.637 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.633+0000 7f610415d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:23:55.662 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:23:55.716 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:55.714+0000 7f7816bff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:23:55.779 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.775+0000 7fd784199700 1 Processor -- start 2023-11-24T06:23:55.779 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.776+0000 7fd784199700 1 -- start start 2023-11-24T06:23:55.781 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd784199700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd77c06c710 0x7fd77c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:55.781 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd784199700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd77c144310 0x7fd77c0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:55.781 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd784199700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c0b0c20 con 0x7fd77c143f30 2023-11-24T06:23:55.781 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd784199700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0b3cb0 con 0x7fd77c144310 2023-11-24T06:23:55.781 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd784199700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0b0e50 con 0x7fd77c06c710 2023-11-24T06:23:55.782 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd781f35700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd77c143f30 0x7fd77c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52838/0 (socket says 172.21.15.114:52838) 2023-11-24T06:23:55.782 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd781f35700 1 -- 172.21.15.114:0/1185986850 learned_addr learned my addr 172.21.15.114:0/1185986850 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:23:55.782 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 --2- 172.21.15.114:0/1185986850 >> v2:172.21.15.114:3300/0 conn(0x7fd77c144310 0x7fd77c0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:55.782 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd781734700 1 --2- 172.21.15.114:0/1185986850 >> v2:172.21.15.172:3300/0 conn(0x7fd77c06c710 0x7fd77c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:55.782 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 -- 172.21.15.114:0/1185986850 >> v2:172.21.15.172:3300/0 conn(0x7fd77c06c710 msgr2=0x7fd77c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 --2- 172.21.15.114:0/1185986850 >> v2:172.21.15.172:3300/0 conn(0x7fd77c06c710 0x7fd77c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 -- 172.21.15.114:0/1185986850 >> v1:172.21.15.114:6789/0 conn(0x7fd77c143f30 legacy=0x7fd77c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd780f33700 1 -- 172.21.15.114:0/1185986850 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683664651 0 0) 0x7fd77c0b0c20 con 0x7fd77c143f30 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 -- 172.21.15.114:0/1185986850 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0b1130 con 0x7fd77c144310 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd782736700 1 --2- 172.21.15.114:0/1185986850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c0b06e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd77c062680 tx=0x7fd77400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23ad1e91e6eb2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:55.783 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd780f33700 1 -- 172.21.15.114:0/1185986850 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400e070 con 0x7fd77c144310 2023-11-24T06:23:55.784 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd780f33700 1 -- 172.21.15.114:0/1185986850 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77400bb20 con 0x7fd77c144310 2023-11-24T06:23:55.784 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.777+0000 7fd780f33700 1 -- 172.21.15.114:0/1185986850 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774012430 con 0x7fd77c144310 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 -- 172.21.15.114:0/1185986850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 msgr2=0x7fd77c0b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 --2- 172.21.15.114:0/1185986850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c0b06e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd77c062680 tx=0x7fd77400b330 comp rx=0 tx=0).stop 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 -- 172.21.15.114:0/1185986850 shutdown_connections 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 --2- 172.21.15.114:0/1185986850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c0b06e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 --2- 172.21.15.114:0/1185986850 >> v2:172.21.15.172:3300/0 conn(0x7fd77c06c710 0x7fd77c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 -- 172.21.15.114:0/1185986850 >> 172.21.15.114:0/1185986850 conn(0x7fd77c071a50 msgr2=0x7fd77c071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 -- 172.21.15.114:0/1185986850 shutdown_connections 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.778+0000 7fd784199700 1 -- 172.21.15.114:0/1185986850 wait complete. 2023-11-24T06:23:55.787 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.779+0000 7fd784199700 1 Processor -- start 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.779+0000 7fd784199700 1 -- start start 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd77c06c710 0x7fd77c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77c143f30 0x7fd77c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c15d710 con 0x7fd77c144310 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0ce6f0 con 0x7fd77c143f30 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd784199700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c15c550 con 0x7fd77c06c710 2023-11-24T06:23:55.788 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd782736700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd782736700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37140/0 (socket says 172.21.15.114:37140) 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd782736700 1 -- 172.21.15.114:0/2438029458 learned_addr learned my addr 172.21.15.114:0/2438029458 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd781734700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77c143f30 0x7fd77c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.780+0000 7fd781f35700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd77c06c710 0x7fd77c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 -- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd77c06c710 msgr2=0x7fd77c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd77c06c710 0x7fd77c15cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 -- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77c143f30 msgr2=0x7fd77c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77c143f30 0x7fd77c13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.789 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 -- 172.21.15.114:0/2438029458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774009a70 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd782736700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd774013000 tx=0x7fd774004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774012430 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd774020070 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c15d150 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774004280 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c0ce900 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd7740046e0 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.781+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd774022070 con 0x7fd77c144310 2023-11-24T06:23:55.790 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.782+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c068fb0 con 0x7fd77c144310 2023-11-24T06:23:55.791 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.784+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd774012e20 con 0x7fd77c144310 2023-11-24T06:23:55.854 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.850+0000 7f610415d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:23:55.858 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.855+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd77c06f8e0 con 0x7fd77c144310 2023-11-24T06:23:55.859 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.856+0000 7fd7727fc700 1 -- 172.21.15.114:0/2438029458 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd774030420 con 0x7fd77c144310 2023-11-24T06:23:55.859 INFO:teuthology.misc.health.smithi114.stdout: 2023-11-24T06:23:55.859 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":10,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:23:54.578009+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:23:55.860 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 msgr2=0x7fd77c15c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:23:55.860 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd774013000 tx=0x7fd774004ad0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 shutdown_connections 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd77c144310 0x7fd77c15c010 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd77c06c710 0x7fd77c15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 --2- 172.21.15.114:0/2438029458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77c143f30 0x7fd77c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 >> 172.21.15.114:0/2438029458 conn(0x7fd77c071a50 msgr2=0x7fd77c1598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 shutdown_connections 2023-11-24T06:23:55.861 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:23:55.857+0000 7fd784199700 1 -- 172.21.15.114:0/2438029458 wait complete. 2023-11-24T06:23:55.871 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:23:55.931 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:55.928+0000 7f7816bff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:23:55.988 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:55.984+0000 7f610415d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:23:56.064 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.062+0000 7f7816bff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:23:56.208 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:56.205+0000 7f610415d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:23:56.283 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.280+0000 7f7816bff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:23:56.363 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:56.360+0000 7f610415d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:23:56.438 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.436+0000 7f7816bff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:23:56.478 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:56.475+0000 7f610415d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:23:56.552 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.550+0000 7f7816bff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:23:56.799 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:56.796+0000 7f610415d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:23:56.879 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.877+0000 7f7816bff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:23:56.901 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:56.897+0000 7f610415d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:23:56.978 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:56.976+0000 7f7816bff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:23:57.087 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:57.084+0000 7f610415d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:23:57.167 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:57.165+0000 7f7816bff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:23:57.687 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:57.684+0000 7f610415d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:23:57.774 INFO:tasks.ceph.mgr.x.smithi172.stderr:2023-11-24T06:23:57.771+0000 7f7816bff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:23:58.711 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:58.708+0000 7f60bc0eb700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:23:58.711 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:23:58.708+0000 7f60bc0eb700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:23:58.730 INFO:tasks.ceph.osd.1.smithi114.stderr:2023-11-24T06:23:58.724+0000 7fee8ce1e640 -1 osd.1 0 log_to_monitors true 2023-11-24T06:23:58.746 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-11-24T06:23:58.744+0000 7f97c53e4640 -1 osd.4 0 log_to_monitors true 2023-11-24T06:23:58.750 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-11-24T06:23:58.748+0000 7fd4c4213640 -1 osd.5 0 log_to_monitors true 2023-11-24T06:23:58.758 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-11-24T06:23:58.752+0000 7fc01fb02640 -1 osd.6 0 log_to_monitors true 2023-11-24T06:23:58.765 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-11-24T06:23:58.763+0000 7fdc9e901640 -1 osd.7 0 log_to_monitors true 2023-11-24T06:23:58.775 INFO:tasks.ceph.osd.0.smithi114.stderr:2023-11-24T06:23:58.772+0000 7fc59a085640 -1 osd.0 0 log_to_monitors true 2023-11-24T06:23:58.789 INFO:tasks.ceph.osd.2.smithi114.stderr:2023-11-24T06:23:58.786+0000 7faa8e786640 -1 osd.2 0 log_to_monitors true 2023-11-24T06:23:58.800 INFO:tasks.ceph.osd.3.smithi114.stderr:2023-11-24T06:23:58.797+0000 7fe64a58c640 -1 osd.3 0 log_to_monitors true 2023-11-24T06:24:02.677 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:02.736 INFO:tasks.ceph.osd.3.smithi114.stderr:2023-11-24T06:24:02.732+0000 7fe63e143700 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:24:02.752 INFO:tasks.ceph.osd.3.smithi114.stderr:2023-11-24T06:24:02.749+0000 7fe636523700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.770 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-11-24T06:24:02.768+0000 7fdc924b8700 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:24:02.773 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-11-24T06:24:02.771+0000 7fd4b7dca700 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:24:02.777 INFO:tasks.ceph.osd.2.smithi114.stderr:2023-11-24T06:24:02.774+0000 7faa8233d700 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:24:02.779 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-11-24T06:24:02.777+0000 7fc0136b9700 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:24:02.782 INFO:tasks.ceph.osd.1.smithi114.stderr:2023-11-24T06:24:02.780+0000 7fee821d8700 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:24:02.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2023-11-24T06:24:02.782+0000 7fc58dc3c700 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:24:02.788 INFO:tasks.ceph.osd.2.smithi114.stderr:2023-11-24T06:24:02.786+0000 7faa7a71d700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.789 INFO:tasks.ceph.osd.5.smithi172.stderr:2023-11-24T06:24:02.787+0000 7fd4b01aa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.790 INFO:tasks.ceph.osd.7.smithi172.stderr:2023-11-24T06:24:02.788+0000 7fdc8a898700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.793 INFO:tasks.ceph.osd.6.smithi172.stderr:2023-11-24T06:24:02.791+0000 7fc00ba99700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.795 INFO:tasks.ceph.osd.1.smithi114.stderr:2023-11-24T06:24:02.793+0000 7fee78db5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.796 INFO:tasks.ceph.osd.0.smithi114.stderr:2023-11-24T06:24:02.794+0000 7fc58601c700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.813 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-11-24T06:24:02.811+0000 7f97b8f9b700 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:24:02.822 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.820+0000 7f47ca184700 1 Processor -- start 2023-11-24T06:24:02.822 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.820+0000 7f47ca184700 1 -- start start 2023-11-24T06:24:02.824 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.820+0000 7f47ca184700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 0x7f47c406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:02.824 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47ca184700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c40cfe50 con 0x7f47c406bf40 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47ca184700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c40d0130 con 0x7f47c40d2280 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47ca184700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c415e5f0 con 0x7f47c415e210 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c2ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f47c406bf40 0x7f47c40d1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52950/0 (socket says 172.21.15.114:52950) 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c2ffd700 1 -- 172.21.15.114:0/3831886727 learned_addr learned my addr 172.21.15.114:0/3831886727 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c37fe700 1 --2- 172.21.15.114:0/3831886727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 0x7f47c406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473841125 0 0) 0x7f47c40cfe50 con 0x7f47c406bf40 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ac0035b0 con 0x7f47c406bf40 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160242579 0 0) 0x7f47c40d0130 con 0x7f47c40d2280 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c40cfe50 con 0x7f47c40d2280 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c37fe700 1 -- 172.21.15.114:0/3831886727 >> v1:172.21.15.172:6789/0 conn(0x7f47c40d2280 legacy=0x7f47c40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:02.825 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c37fe700 1 -- 172.21.15.114:0/3831886727 >> v1:172.21.15.114:6789/0 conn(0x7f47c406bf40 legacy=0x7f47c40d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c37fe700 1 -- 172.21.15.114:0/3831886727 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40d0410 con 0x7f47c415e210 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c37fe700 1 --2- 172.21.15.114:0/3831886727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 0x7f47c406ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009870 tx=0x7f47b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5b9cedbfccf9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011830 con 0x7f47c415e210 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4011990 con 0x7f47c415e210 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.821+0000 7f47c27fc700 1 -- 172.21.15.114:0/3831886727 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011d20 con 0x7f47c415e210 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 -- 172.21.15.114:0/3831886727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 msgr2=0x7f47c406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 --2- 172.21.15.114:0/3831886727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 0x7f47c406ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009870 tx=0x7f47b4009b80 comp rx=0 tx=0).stop 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 -- 172.21.15.114:0/3831886727 shutdown_connections 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 --2- 172.21.15.114:0/3831886727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c415e210 0x7f47c406ba00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 -- 172.21.15.114:0/3831886727 >> 172.21.15.114:0/3831886727 conn(0x7f47c409aa00 msgr2=0x7f47c409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 -- 172.21.15.114:0/3831886727 shutdown_connections 2023-11-24T06:24:02.826 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.822+0000 7f47ca184700 1 -- 172.21.15.114:0/3831886727 wait complete. 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.823+0000 7f47ca184700 1 Processor -- start 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.823+0000 7f47ca184700 1 -- start start 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406bf40 0x7f47c40c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c415e210 0x7f47c40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:02.829 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b5050 con 0x7f47c40d2280 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40c7f10 con 0x7f47c415e210 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47ca184700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b4990 con 0x7f47c406bf40 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37246/0 (socket says 172.21.15.114:37246) 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 -- 172.21.15.114:0/1461296210 learned_addr learned my addr 172.21.15.114:0/1461296210 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c37fe700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406bf40 0x7f47c40c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c3fff700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c415e210 0x7f47c40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 -- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406bf40 msgr2=0x7f47c40c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406bf40 0x7f47c40c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 -- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c415e210 msgr2=0x7f47c40af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:02.830 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c415e210 0x7f47c40af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 -- 172.21.15.114:0/1461296210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40094a0 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.824+0000 7f47c2ffd700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005910 tx=0x7f47b800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.825+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8005d70 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.825+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80070d0 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.825+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40d1070 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.825+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8007460 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.825+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40c7b50 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.826+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40b47a0 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.826+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f47b80075c0 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.826+0000 7f47abfff700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 0x7f47b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.826+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f47b8087bb0 con 0x7f47c40d2280 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.826+0000 7f47c37fe700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 0x7f47b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:02.831 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.827+0000 7f47c37fe700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 0x7f47b006d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47b40090b0 tx=0x7f47b401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:02.832 INFO:tasks.ceph.osd.4.smithi172.stderr:2023-11-24T06:24:02.829+0000 7f97b137b700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:24:02.833 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:02.831+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47b8053ad0 con 0x7f47c40d2280 2023-11-24T06:24:03.025 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.023+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47c4068fb0 con 0x7f47c40d2280 2023-11-24T06:24:03.025 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.023+0000 7f47abfff700 1 -- 172.21.15.114:0/1461296210 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f47b8017020 con 0x7f47c40d2280 2023-11-24T06:24:03.026 INFO:teuthology.misc.health.smithi114.stdout: 2023-11-24T06:24:03.026 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":12,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:00.713734+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 msgr2=0x7f47b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 0x7f47b006d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47b40090b0 tx=0x7f47b401c040 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 msgr2=0x7f47c40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005910 tx=0x7f47b800c9e0 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 shutdown_connections 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47b006ae40 0x7f47b006d300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47c415e210 0x7f47c40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47c406bf40 0x7f47c40c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 --2- 172.21.15.114:0/1461296210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47c40d2280 0x7f47c40aeda0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 >> 172.21.15.114:0/1461296210 conn(0x7f47c409aa00 msgr2=0x7f47c406edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:03.028 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 shutdown_connections 2023-11-24T06:24:03.029 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:03.025+0000 7f47ca184700 1 -- 172.21.15.114:0/1461296210 wait complete. 2023-11-24T06:24:03.040 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:24:09.847 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:09.969 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.966+0000 7fdf0ce8a700 1 Processor -- start 2023-11-24T06:24:09.969 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.966+0000 7fdf0ce8a700 1 -- start start 2023-11-24T06:24:09.978 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf0ce8a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdf080d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.978 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf0ce8a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080b80e0 con 0x7fdf080d39d0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf0ce8a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080b67d0 con 0x7fdf080b98c0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf0ce8a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0815e520 con 0x7fdf0815e140 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf05d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdf080d39d0 0x7fdf080b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53180/0 (socket says 172.21.15.114:53180) 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.967+0000 7fdf05d9b700 1 -- 172.21.15.114:0/3573326213 learned_addr learned my addr 172.21.15.114:0/3573326213 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0659c700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdf080d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250189899 0 0) 0x7fdef8001110 con 0x7fdf080d39d0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef00035b0 con 0x7fdf080d39d0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380670704 0 0) 0x7fdef00035b0 con 0x7fdf080d39d0 2023-11-24T06:24:09.979 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 msgr2=0x7fdf080d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdf080d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 >> v1:172.21.15.172:6789/0 conn(0x7fdf080b98c0 legacy=0x7fdf080b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080d2930 con 0x7fdf080d39d0 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fdef8001800 con 0x7fdf080d39d0 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 >> v1:172.21.15.114:6789/0 conn(0x7fdf080d39d0 legacy=0x7fdf080b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:09.980 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef0003a30 0x7fdef0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdef00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdef0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0815e520 con 0x7fdef00043e0 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000ba10 con 0x7fdef0003a30 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000bbd0 con 0x7fdf0815e140 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdef00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.981 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf0659c700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef0003a30 0x7fdef0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf06d9d700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdef0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 -- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 msgr2=0x7fdef0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf0815e140 0x7fdef0005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 -- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef0003a30 msgr2=0x7fdef0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef0003a30 0x7fdef0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:09.982 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 -- 172.21.15.114:0/3573326213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00035b0 con 0x7fdef00043e0 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.968+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdef00048b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdef8005140 tx=0x7fdef0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933d5f9777aa8ade server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8005b60 con 0x7fdef00043e0 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8005cc0 con 0x7fdef00043e0 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0559a700 1 -- 172.21.15.114:0/3573326213 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8008320 con 0x7fdef00043e0 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/3573326213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 msgr2=0x7fdef00048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/3573326213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdef00048b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdef8005140 tx=0x7fdef0005880 comp rx=0 tx=0).stop 2023-11-24T06:24:09.983 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf06d9d700 1 -- 172.21.15.114:0/3573326213 reap_dead start 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/3573326213 shutdown_connections 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/3573326213 >> 172.21.15.114:0/3573326213 conn(0x7fdf0806b410 msgr2=0x7fdf0806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/3573326213 shutdown_connections 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.969+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/3573326213 wait complete. 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 Processor -- start 2023-11-24T06:24:09.984 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 -- start start 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 0x7fdf080b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf08144210 0x7fdf08144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080b60b0 con 0x7fdef00043e0 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080ca5d0 con 0x7fdf08144210 2023-11-24T06:24:09.985 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0ce8a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf081f3ab0 con 0x7fdf080b56f0 2023-11-24T06:24:09.986 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.986 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37480/0 (socket says 172.21.15.114:37480) 2023-11-24T06:24:09.986 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.971+0000 7fdf0659c700 1 -- 172.21.15.114:0/2173380050 learned_addr learned my addr 172.21.15.114:0/2173380050 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:09.986 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 0x7fdf080b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.986 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf06d9d700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf08144210 0x7fdf08144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 -- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 msgr2=0x7fdf080b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 0x7fdf080b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 -- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf08144210 msgr2=0x7fdf08144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf08144210 0x7fdf08144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 -- 172.21.15.114:0/2173380050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080c99c0 con 0x7fdef00043e0 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 0x7fdf080b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:09.987 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0659c700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdefc011ad0 tx=0x7fdefc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00f9a0 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc00fb00 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf080c9ca0 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00feb0 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.972+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf080ca260 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.973+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf08068fb0 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.974+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdefc014440 con 0x7fdef00043e0 2023-11-24T06:24:09.988 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.974+0000 7fdef6ffd700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 0x7fdee806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:09.989 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.974+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdefc089a70 con 0x7fdef00043e0 2023-11-24T06:24:09.989 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.974+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 0x7fdee806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:09.989 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.974+0000 7fdf05d9b700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 0x7fdee806d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf081f7150 tx=0x7fdef800a070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:09.990 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:09.979+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdefc054e90 con 0x7fdef00043e0 2023-11-24T06:24:10.172 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.170+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf080723e0 con 0x7fdef00043e0 2023-11-24T06:24:10.173 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.171+0000 7fdef6ffd700 1 -- 172.21.15.114:0/2173380050 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fdefc086270 con 0x7fdef00043e0 2023-11-24T06:24:10.173 INFO:teuthology.misc.health.smithi114.stdout: 2023-11-24T06:24:10.173 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":17,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:07.882684+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 msgr2=0x7fdee806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 0x7fdee806d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf081f7150 tx=0x7fdef800a070 comp rx=0 tx=0).stop 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 msgr2=0x7fdf08160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdefc011ad0 tx=0x7fdefc011e90 comp rx=0 tx=0).stop 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 shutdown_connections 2023-11-24T06:24:10.176 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdee806aed0 0x7fdee806d390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf08144210 0x7fdf08144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdef00043e0 0x7fdf08160ad0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 --2- 172.21.15.114:0/2173380050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf080b56f0 0x7fdf080b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 >> 172.21.15.114:0/2173380050 conn(0x7fdf0806b410 msgr2=0x7fdf080b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 shutdown_connections 2023-11-24T06:24:10.177 INFO:teuthology.misc.health.smithi114.stderr:2023-11-24T06:24:10.173+0000 7fdf0ce8a700 1 -- 172.21.15.114:0/2173380050 wait complete. 2023-11-24T06:24:10.187 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:24:10.187 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:24:10.188 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:24:10.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:24:10.190 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:24:10.190 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:24:10.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:24:10.193 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:24:10.193 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:24:10.193 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:24:10.196 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:24:10.196 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:24:10.196 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:24:10.199 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:24:10.199 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:24:10.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:10.269 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2023-11-24T06:24:10.269 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:24:10.270 INFO:tasks.ceph.mds.c.smithi114.stderr:2023-11-24T06:24:10.267+0000 7fb80ec1b700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T06:24:10.287 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-11-24T06:24:10.287 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-11-24T06:24:10.288 INFO:tasks.ceph.mds.b.smithi172.stderr:2023-11-24T06:24:10.286+0000 7f6ae3222700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2023-11-24T06:24:10.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.323+0000 7f24ad163700 1 Processor -- start 2023-11-24T06:24:10.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.323+0000 7f24ad163700 1 -- start start 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24ad163700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24a815e1e0 0x7f24a80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24ad163700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a80d0f80 0x7f24a80b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24ad163700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a80d20b0 con 0x7f24a80d3a00 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24ad163700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a80d3150 con 0x7f24a815e1e0 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24ad163700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a815e5c0 con 0x7f24a80d0f80 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a659c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f24a80d3a00 0x7f24a80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53222/0 (socket says 172.21.15.114:53222) 2023-11-24T06:24:10.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24a815e1e0 0x7f24a80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a659c700 1 -- 172.21.15.114:0/255760288 learned_addr learned my addr 172.21.15.114:0/255760288 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a759e700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a80d0f80 0x7f24a80b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 -- 172.21.15.114:0/255760288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a80d0f80 msgr2=0x7f24a80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a80d0f80 0x7f24a80b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 -- 172.21.15.114:0/255760288 >> v1:172.21.15.114:6789/0 conn(0x7f24a80d3a00 legacy=0x7f24a80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a5d9b700 1 -- 172.21.15.114:0/255760288 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908212336 0 0) 0x7f24a80d20b0 con 0x7f24a80d3a00 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 -- 172.21.15.114:0/255760288 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80d2390 con 0x7f24a815e1e0 2023-11-24T06:24:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.324+0000 7f24a6d9d700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a815e1e0 0x7f24a80d34c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2498009870 tx=0x7f2498009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97932a74220803 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24a5d9b700 1 -- 172.21.15.114:0/255760288 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011830 con 0x7f24a815e1e0 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24a5d9b700 1 -- 172.21.15.114:0/255760288 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498011990 con 0x7f24a815e1e0 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24a5d9b700 1 -- 172.21.15.114:0/255760288 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011d20 con 0x7f24a815e1e0 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 -- 172.21.15.114:0/255760288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a815e1e0 msgr2=0x7f24a80d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a815e1e0 0x7f24a80d34c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2498009870 tx=0x7f2498009b80 comp rx=0 tx=0).stop 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 -- 172.21.15.114:0/255760288 shutdown_connections 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a80d0f80 0x7f24a80b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 --2- 172.21.15.114:0/255760288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a815e1e0 0x7f24a80d34c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 -- 172.21.15.114:0/255760288 >> 172.21.15.114:0/255760288 conn(0x7f24a80b5b50 msgr2=0x7f24a80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:10.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 -- 172.21.15.114:0/255760288 shutdown_connections 2023-11-24T06:24:10.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.325+0000 7f24ad163700 1 -- 172.21.15.114:0/255760288 wait complete. 2023-11-24T06:24:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 Processor -- start 2023-11-24T06:24:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 -- start start 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a80d3a00 0x7f24a814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a815e1e0 0x7f24a80acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80ab480 con 0x7f24a80d0f80 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80ab760 con 0x7f24a80d3a00 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24ad163700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80aba40 con 0x7f24a815e1e0 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.327+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37518/0 (socket says 172.21.15.114:37518) 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 -- 172.21.15.114:0/2283482007 learned_addr learned my addr 172.21.15.114:0/2283482007 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a659c700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a80d3a00 0x7f24a814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a759e700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a815e1e0 0x7f24a80acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 -- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a815e1e0 msgr2=0x7f24a80acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a815e1e0 0x7f24a80acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 -- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a80d3a00 msgr2=0x7f24a814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a80d3a00 0x7f24a814f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 -- 172.21.15.114:0/2283482007 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980094a0 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24a6d9d700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24980090b0 tx=0x7f2498004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498004590 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249801e070 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a80abd20 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980038c0 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.328+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a814f6c0 con 0x7f24a80d0f80 2023-11-24T06:24:10.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.329+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a80751e0 con 0x7f24a80d0f80 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.329+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2498003a20 con 0x7f24a80d0f80 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.330+0000 7f248f7fe700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 0x7f249406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.330+0000 7f24a659c700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 0x7f249406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.330+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f249808a1c0 con 0x7f24a80d0f80 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.330+0000 7f24a659c700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 0x7f249406d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24a8152f20 tx=0x7f249c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.335+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24980555c0 con 0x7f24a80d0f80 2023-11-24T06:24:10.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.525+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24a80b2ad0 con 0x7f24a80d0f80 2023-11-24T06:24:10.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.526+0000 7f248f7fe700 1 -- 172.21.15.114:0/2283482007 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f2498088e60 con 0x7f24a80d0f80 2023-11-24T06:24:10.530 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:10.530 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:07.882684+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:10.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 msgr2=0x7f249406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 0x7f249406d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24a8152f20 tx=0x7f249c007de0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 msgr2=0x7f24a80ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24980090b0 tx=0x7f2498004750 comp rx=0 tx=0).stop 2023-11-24T06:24:10.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 shutdown_connections 2023-11-24T06:24:10.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f249406aed0 0x7f249406d390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24a815e1e0 0x7f24a80acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24a80d0f80 0x7f24a80ac520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 --2- 172.21.15.114:0/2283482007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a80d3a00 0x7f24a814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 >> 172.21.15.114:0/2283482007 conn(0x7f24a80b5b50 msgr2=0x7f24a80ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:10.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 shutdown_connections 2023-11-24T06:24:10.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.528+0000 7f24ad163700 1 -- 172.21.15.114:0/2283482007 wait complete. 2023-11-24T06:24:10.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:10.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:10.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.664+0000 7f56c099c700 1 Processor -- start 2023-11-24T06:24:10.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.664+0000 7f56c099c700 1 -- start start 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56c099c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56bc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56c099c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56bc0afc90 0x7f56bc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56c099c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc0b3c60 con 0x7f56bc15e160 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56c099c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56bc0b4500 con 0x7f56bc0afc90 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56c099c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b58c0 con 0x7f56bc0746a0 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56ba59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f56bc15e160 0x7f56bc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53234/0 (socket says 172.21.15.114:53234) 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56ba59c700 1 -- 172.21.15.114:0/2337437819 learned_addr learned my addr 172.21.15.114:0/2337437819 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:10.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b9d9b700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56bc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56bad9d700 1 --2- 172.21.15.114:0/2337437819 >> v2:172.21.15.172:3300/0 conn(0x7f56bc0afc90 0x7f56bc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894806787 0 0) 0x7f56bc0b3c60 con 0x7f56bc15e160 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a40035b0 con 0x7f56bc15e160 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627343092 0 0) 0x7f56a40035b0 con 0x7f56bc15e160 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 msgr2=0x7f56bc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56bc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 >> v2:172.21.15.172:3300/0 conn(0x7f56bc0afc90 msgr2=0x7f56bc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 --2- 172.21.15.114:0/2337437819 >> v2:172.21.15.172:3300/0 conn(0x7f56bc0afc90 0x7f56bc06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0b0140 con 0x7f56bc15e160 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f56ac001670 con 0x7f56bc15e160 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 >> v1:172.21.15.114:6789/0 conn(0x7f56bc15e160 legacy=0x7f56bc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56a4003a00 0x7f56a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a4004850 0x7f56a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b4500 con 0x7f56a4003a00 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b58c0 con 0x7f56a4004850 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.665+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a400ba10 con 0x7f56bc0746a0 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56a4003a00 0x7f56a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56bad9d700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56b9d9b700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a4004850 0x7f56a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 -- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 msgr2=0x7f56a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:10.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56bc0746a0 0x7f56a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 -- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a4004850 msgr2=0x7f56a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a4004850 0x7f56a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 -- 172.21.15.114:0/2337437819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40035b0 con 0x7f56a4003a00 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56ba59c700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56a4003a00 0x7f56a4003e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c2f0 tx=0x7f56a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c590d43a3a08e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00ce30 con 0x7f56a4003a00 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac005960 con 0x7f56a4003a00 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.666+0000 7f56b959a700 1 -- 172.21.15.114:0/2337437819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac005d10 con 0x7f56a4003a00 2023-11-24T06:24:10.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 -- 172.21.15.114:0/2337437819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56a4003a00 msgr2=0x7f56a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 --2- 172.21.15.114:0/2337437819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56a4003a00 0x7f56a4003e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c2f0 tx=0x7f56a4005880 comp rx=0 tx=0).stop 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56bad9d700 1 -- 172.21.15.114:0/2337437819 reap_dead start 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 -- 172.21.15.114:0/2337437819 shutdown_connections 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 -- 172.21.15.114:0/2337437819 >> 172.21.15.114:0/2337437819 conn(0x7f56bc0d0bf0 msgr2=0x7f56bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 -- 172.21.15.114:0/2337437819 shutdown_connections 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.667+0000 7f56c099c700 1 -- 172.21.15.114:0/2337437819 wait complete. 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.668+0000 7f56c099c700 1 Processor -- start 2023-11-24T06:24:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.668+0000 7f56c099c700 1 -- start start 2023-11-24T06:24:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56a4003a00 0x7f56bc160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc1579b0 0x7f56bc1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc1f7650 con 0x7f56bc1f71d0 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc1483f0 con 0x7f56bc1579b0 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc1486d0 con 0x7f56a4003a00 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37542/0 (socket says 172.21.15.114:37542) 2023-11-24T06:24:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 -- 172.21.15.114:0/1675676203 learned_addr learned my addr 172.21.15.114:0/1675676203 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56b9d9b700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc1579b0 0x7f56bc1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56ba59c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56a4003a00 0x7f56bc160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 -- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56a4003a00 msgr2=0x7f56bc160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56a4003a00 0x7f56bc160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 -- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc1579b0 msgr2=0x7f56bc1f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc1579b0 0x7f56bc1f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 -- 172.21.15.114:0/1675676203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc1489b0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56bad9d700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0b5070 tx=0x7f56b4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400e040 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b400f040 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc148c90 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400fdb0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.669+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc161290 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.670+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc1492e0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.671+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56b4022460 con 0x7f56bc1f71d0 2023-11-24T06:24:10.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.671+0000 7f56aaffd700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:10.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.671+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56b40897f0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.671+0000 7f56ba59c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:10.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.671+0000 7f56ba59c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 0x7f569c06d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008fd0 tx=0x7f56ac00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:10.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.675+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56b4054bf0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.866+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56bc068fb0 con 0x7f56bc1f71d0 2023-11-24T06:24:10.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.866+0000 7f56aaffd700 1 -- 172.21.15.114:0/1675676203 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56b4058210 con 0x7f56bc1f71d0 2023-11-24T06:24:10.868 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 msgr2=0x7f569c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 0x7f569c06d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008fd0 tx=0x7f56ac00a000 comp rx=0 tx=0).stop 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 msgr2=0x7f56bc155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0b5070 tx=0x7f56b4011f80 comp rx=0 tx=0).stop 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 shutdown_connections 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56bc1f71d0 0x7f56bc155f90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56a4003a00 0x7f56bc160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 --2- 172.21.15.114:0/1675676203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc1579b0 0x7f56bc1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:10.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 >> 172.21.15.114:0/1675676203 conn(0x7f56bc0d0bf0 msgr2=0x7f56bc1619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:10.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 shutdown_connections 2023-11-24T06:24:10.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:10.868+0000 7f56c099c700 1 -- 172.21.15.114:0/1675676203 wait complete. 2023-11-24T06:24:10.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:11.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.000+0000 7fcc8358e700 1 Processor -- start 2023-11-24T06:24:11.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.000+0000 7fcc8358e700 1 -- start start 2023-11-24T06:24:11.009 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8358e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcc7c15e080 0x7fcc7c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8358e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc7c0af4d0 0x7fcc7c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8358e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0ae430 con 0x7fcc7c06c000 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8358e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0af0b0 con 0x7fcc7c15e080 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8358e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0b05a0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcc7c15e080 0x7fcc7c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc81b2b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcc7c06c000 0x7fcc7c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53258/0 (socket says 172.21.15.114:53258) 2023-11-24T06:24:11.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc81b2b700 1 -- 172.21.15.114:0/2093682144 learned_addr learned my addr 172.21.15.114:0/2093682144 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:11.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc80b29700 1 --2- 172.21.15.114:0/2093682144 >> v2:172.21.15.172:3300/0 conn(0x7fcc7c0af4d0 0x7fcc7c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 -- 172.21.15.114:0/2093682144 >> v2:172.21.15.172:3300/0 conn(0x7fcc7c0af4d0 msgr2=0x7fcc7c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:11.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 --2- 172.21.15.114:0/2093682144 >> v2:172.21.15.172:3300/0 conn(0x7fcc7c0af4d0 0x7fcc7c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 -- 172.21.15.114:0/2093682144 >> v1:172.21.15.114:6789/0 conn(0x7fcc7c06c000 legacy=0x7fcc7c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc73fff700 1 -- 172.21.15.114:0/2093682144 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811748506 0 0) 0x7fcc7c0ae430 con 0x7fcc7c06c000 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 -- 172.21.15.114:0/2093682144 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0ae710 con 0x7fcc7c15e080 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc8132a700 1 --2- 172.21.15.114:0/2093682144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c15e080 0x7fcc7c0b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06e4e0 tx=0x7fcc78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40221a6e7d63362d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.001+0000 7fcc73fff700 1 -- 172.21.15.114:0/2093682144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78011830 con 0x7fcc7c15e080 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc73fff700 1 -- 172.21.15.114:0/2093682144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78011990 con 0x7fcc7c15e080 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc73fff700 1 -- 172.21.15.114:0/2093682144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78011d20 con 0x7fcc7c15e080 2023-11-24T06:24:11.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 -- 172.21.15.114:0/2093682144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c15e080 msgr2=0x7fcc7c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 --2- 172.21.15.114:0/2093682144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c15e080 0x7fcc7c0b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06e4e0 tx=0x7fcc78009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 -- 172.21.15.114:0/2093682144 shutdown_connections 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 --2- 172.21.15.114:0/2093682144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c15e080 0x7fcc7c0b5050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 --2- 172.21.15.114:0/2093682144 >> v2:172.21.15.172:3300/0 conn(0x7fcc7c0af4d0 0x7fcc7c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 -- 172.21.15.114:0/2093682144 >> 172.21.15.114:0/2093682144 conn(0x7fcc7c071b90 msgr2=0x7fcc7c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 -- 172.21.15.114:0/2093682144 shutdown_connections 2023-11-24T06:24:11.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.002+0000 7fcc8358e700 1 -- 172.21.15.114:0/2093682144 wait complete. 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.003+0000 7fcc8358e700 1 Processor -- start 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.003+0000 7fcc8358e700 1 -- start start 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc7c06c000 0x7fcc7c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc7c15e080 0x7fcc7c149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c14a950 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1542b0 con 0x7fcc7c15e080 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8358e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c158e50 con 0x7fcc7c06c000 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37554/0 (socket says 172.21.15.114:37554) 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 -- 172.21.15.114:0/3606590144 learned_addr learned my addr 172.21.15.114:0/3606590144 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc81b2b700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc7c15e080 0x7fcc7c149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc8132a700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc7c06c000 0x7fcc7c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 -- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc7c06c000 msgr2=0x7fcc7c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc7c06c000 0x7fcc7c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 -- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc7c15e080 msgr2=0x7fcc7c149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc7c15e080 0x7fcc7c149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.004+0000 7fcc80b29700 1 -- 172.21.15.114:0/3606590144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780094a0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc80b29700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e7e0 tx=0x7fcc6c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c013610 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c013770 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c013ae0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c159130 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.005+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c1547e0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.006+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c0753f0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.006+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc6c013c40 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.007+0000 7fcc71ffb700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:11.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.007+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc6c088f00 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.007+0000 7fcc8132a700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:11.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.007+0000 7fcc8132a700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097c0 tx=0x7fcc7801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:11.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.011+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc6c054390 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.241+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc7c0b00d0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.242+0000 7fcc71ffb700 1 -- 172.21.15.114:0/3606590144 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7fcc6c0579b0 con 0x7fcc7c0af4d0 2023-11-24T06:24:11.245 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:11.245 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 msgr2=0x7fcc6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097c0 tx=0x7fcc7801c040 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 msgr2=0x7fcc7c1494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00e7e0 tx=0x7fcc6c00eba0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 shutdown_connections 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc7c15e080 0x7fcc7c149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc7c06c000 0x7fcc7c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 --2- 172.21.15.114:0/3606590144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc7c0af4d0 0x7fcc7c1494c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 >> 172.21.15.114:0/3606590144 conn(0x7fcc7c071b90 msgr2=0x7fcc7c06f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:11.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 shutdown_connections 2023-11-24T06:24:11.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:11.244+0000 7fcc8358e700 1 -- 172.21.15.114:0/3606590144 wait complete. 2023-11-24T06:24:11.249 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 2 2023-11-24T06:24:13.259 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:16.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:24:16.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:24:16.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.158+0000 7f94027b6700 1 Processor -- start 2023-11-24T06:24:16.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.158+0000 7f94027b6700 1 -- start start 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f94027b6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f94027b6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93fc0af4d0 0x7f93fc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f94027b6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0ae430 con 0x7f93fc06c000 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f94027b6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc0af0b0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f94027b6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0b05a0 con 0x7f93fc15e080 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93fc0af4d0 0x7f93fc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93fc0af4d0 0x7f93fc0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37564/0 (socket says 172.21.15.114:37564) 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 -- 172.21.15.114:0/2926306635 learned_addr learned my addr 172.21.15.114:0/2926306635 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 -- 172.21.15.114:0/2926306635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 msgr2=0x7f93fc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 --2- 172.21.15.114:0/2926306635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 -- 172.21.15.114:0/2926306635 >> v1:172.21.15.172:6789/0 conn(0x7f93fc06c000 legacy=0x7f93fc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 -- 172.21.15.114:0/2926306635 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0ae710 con 0x7f93fc0af4d0 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93fb7fe700 1 --2- 172.21.15.114:0/2926306635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc0afb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009920 tx=0x7f93ec009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53335d7955ea471 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.159+0000 7f93faffd700 1 -- 172.21.15.114:0/2926306635 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004230 con 0x7f93fc0af4d0 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f93faffd700 1 -- 172.21.15.114:0/2926306635 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec004390 con 0x7f93fc0af4d0 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f93faffd700 1 -- 172.21.15.114:0/2926306635 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec009e70 con 0x7f93fc0af4d0 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 -- 172.21.15.114:0/2926306635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 msgr2=0x7f93fc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 --2- 172.21.15.114:0/2926306635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc0afb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009920 tx=0x7f93ec009c30 comp rx=0 tx=0).stop 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 -- 172.21.15.114:0/2926306635 shutdown_connections 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 --2- 172.21.15.114:0/2926306635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 --2- 172.21.15.114:0/2926306635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc0afb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 -- 172.21.15.114:0/2926306635 >> 172.21.15.114:0/2926306635 conn(0x7f93fc071b90 msgr2=0x7f93fc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:16.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 -- 172.21.15.114:0/2926306635 shutdown_connections 2023-11-24T06:24:16.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.160+0000 7f94027b6700 1 -- 172.21.15.114:0/2926306635 wait complete. 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.162+0000 7f94027b6700 1 Processor -- start 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.162+0000 7f94027b6700 1 -- start start 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93fc06c000 0x7f93fc1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1f4fa0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc158640 con 0x7f93fc06c000 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc15a3a0 con 0x7f93fc15e080 2023-11-24T06:24:16.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37570/0 (socket says 172.21.15.114:37570) 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 -- 172.21.15.114:0/482293215 learned_addr learned my addr 172.21.15.114:0/482293215 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fbfff700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93fc06c000 0x7f93fc1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f9400d53700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 -- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 msgr2=0x7f93fc15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 -- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93fc06c000 msgr2=0x7f93fc1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93fc06c000 0x7f93fc1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 -- 172.21.15.114:0/482293215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009550 con 0x7f93fc0af4d0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93fb7fe700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009870 tx=0x7f93ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00be00 con 0x7f93fc0af4d0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec01f070 con 0x7f93fc0af4d0 2023-11-24T06:24:16.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.163+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc15a680 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.164+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec0038c0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.164+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc158b80 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.164+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc0753f0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.165+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93ec021030 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.165+0000 7f93f8ff9700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 0x7f93e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.165+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93ec08a950 con 0x7f93fc0af4d0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.165+0000 7f93fbfff700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 0x7f93e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.166+0000 7f93fbfff700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 0x7f93e806d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.170+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93ec055de0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.404+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93fc0753f0 con 0x7f93fc0af4d0 2023-11-24T06:24:16.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.404+0000 7f93f8ff9700 1 -- 172.21.15.114:0/482293215 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f93ec059400 con 0x7f93fc0af4d0 2023-11-24T06:24:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:16.409 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 msgr2=0x7f93e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 0x7f93e806d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0008040 comp rx=0 tx=0).stop 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 msgr2=0x7f93fc1563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009870 tx=0x7f93ec004750 comp rx=0 tx=0).stop 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 shutdown_connections 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f93e806aed0 0x7f93e806d390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93fc15e080 0x7f93fc15e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93fc06c000 0x7f93fc1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 --2- 172.21.15.114:0/482293215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93fc0af4d0 0x7f93fc1563b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 >> 172.21.15.114:0/482293215 conn(0x7f93fc071b90 msgr2=0x7f93fc0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:16.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 shutdown_connections 2023-11-24T06:24:16.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.406+0000 7f94027b6700 1 -- 172.21.15.114:0/482293215 wait complete. 2023-11-24T06:24:16.411 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 3 2023-11-24T06:24:16.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:16.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 Processor -- start 2023-11-24T06:24:16.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 -- start start 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180b0fb0 0x7fd8180cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8180b1390 0x7fd81806b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8180d0280 0x7fd81806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180b1c60 con 0x7fd8180b1390 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180d0720 con 0x7fd8180d0280 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.538+0000 7fd81d401700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180d0a00 con 0x7fd8180b0fb0 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8180b1390 0x7fd81806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8180b1390 0x7fd81806b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37576/0 (socket says 172.21.15.114:37576) 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 -- 172.21.15.114:0/1220420044 learned_addr learned my addr 172.21.15.114:0/1220420044 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd816ffd700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180b0fb0 0x7fd8180cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8177fe700 1 --2- 172.21.15.114:0/1220420044 >> v2:172.21.15.172:3300/0 conn(0x7fd8180d0280 0x7fd81806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 -- 172.21.15.114:0/1220420044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180b0fb0 msgr2=0x7fd8180cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180b0fb0 0x7fd8180cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 -- 172.21.15.114:0/1220420044 >> v2:172.21.15.172:3300/0 conn(0x7fd8180d0280 msgr2=0x7fd81806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 --2- 172.21.15.114:0/1220420044 >> v2:172.21.15.172:3300/0 conn(0x7fd8180d0280 0x7fd81806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 -- 172.21.15.114:0/1220420044 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180d0ce0 con 0x7fd8180b1390 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd8167fc700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd81806b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8180c8d00 tx=0x7fd80c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5b38eb14806e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd815ffb700 1 -- 172.21.15.114:0/1220420044 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00b870 con 0x7fd8180b1390 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd815ffb700 1 -- 172.21.15.114:0/1220420044 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c00b9d0 con 0x7fd8180b1390 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.539+0000 7fd815ffb700 1 -- 172.21.15.114:0/1220420044 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00bd60 con 0x7fd8180b1390 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 -- 172.21.15.114:0/1220420044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 msgr2=0x7fd81806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd81806b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8180c8d00 tx=0x7fd80c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 -- 172.21.15.114:0/1220420044 shutdown_connections 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180b0fb0 0x7fd8180cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 --2- 172.21.15.114:0/1220420044 >> v2:172.21.15.172:3300/0 conn(0x7fd8180d0280 0x7fd81806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 --2- 172.21.15.114:0/1220420044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd81806b750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 -- 172.21.15.114:0/1220420044 >> 172.21.15.114:0/1220420044 conn(0x7fd81809aa00 msgr2=0x7fd81809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:16.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 -- 172.21.15.114:0/1220420044 shutdown_connections 2023-11-24T06:24:16.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.540+0000 7fd81d401700 1 -- 172.21.15.114:0/1220420044 wait complete. 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.541+0000 7fd81d401700 1 Processor -- start 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.541+0000 7fd81d401700 1 -- start start 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8180b0fb0 0x7fd818151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180d0280 0x7fd8181f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180b76d0 con 0x7fd8180b1390 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180b67f0 con 0x7fd8180b0fb0 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd81d401700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180b6ac0 con 0x7fd8180d0280 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8167fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8167fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37584/0 (socket says 172.21.15.114:37584) 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8167fc700 1 -- 172.21.15.114:0/4255071610 learned_addr learned my addr 172.21.15.114:0/4255071610 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd816ffd700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8180b0fb0 0x7fd818151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8177fe700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180d0280 0x7fd8181f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8167fc700 1 -- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180d0280 msgr2=0x7fd8181f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.542+0000 7fd8167fc700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180d0280 0x7fd8181f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd8167fc700 1 -- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8180b0fb0 msgr2=0x7fd818151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd8167fc700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8180b0fb0 0x7fd818151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd8167fc700 1 -- 172.21.15.114:0/4255071610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c009550 con 0x7fd8180b1390 2023-11-24T06:24:16.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd8167fc700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cf830 tx=0x7fd80c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004310 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c01f070 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8180b6da0 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0036a0 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.543+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8180b7360 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.544+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd818068480 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.544+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd80c021030 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.545+0000 7fd7ff7fe700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 0x7fd80406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.545+0000 7fd816ffd700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 0x7fd80406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.545+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd80c08a5f0 con 0x7fd8180b1390 2023-11-24T06:24:16.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.545+0000 7fd816ffd700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 0x7fd80406d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd80800ab70 tx=0x7fd808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.549+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd80c0559f0 con 0x7fd8180b1390 2023-11-24T06:24:16.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.741+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8180751e0 con 0x7fd8180b1390 2023-11-24T06:24:16.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.742+0000 7fd7ff7fe700 1 -- 172.21.15.114:0/4255071610 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fd80c059010 con 0x7fd8180b1390 2023-11-24T06:24:16.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:16.745 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:07.882684+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:16.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 msgr2=0x7fd80406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 0x7fd80406d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd80800ab70 tx=0x7fd808008040 comp rx=0 tx=0).stop 2023-11-24T06:24:16.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 msgr2=0x7fd8180b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8180cf830 tx=0x7fd80c004750 comp rx=0 tx=0).stop 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 shutdown_connections 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd80406aed0 0x7fd80406d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8180d0280 0x7fd8181f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8180b0fb0 0x7fd818151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 --2- 172.21.15.114:0/4255071610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180b1390 0x7fd8180b62b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 >> 172.21.15.114:0/4255071610 conn(0x7fd81809aa00 msgr2=0x7fd8180b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 shutdown_connections 2023-11-24T06:24:16.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.744+0000 7fd81d401700 1 -- 172.21.15.114:0/4255071610 wait complete. 2023-11-24T06:24:16.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:16.759 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:16.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.881+0000 7fa1e2de1700 1 Processor -- start 2023-11-24T06:24:16.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.881+0000 7fa1e2de1700 1 -- start start 2023-11-24T06:24:16.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e2de1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1dc15bf40 0x7fa1dc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e2de1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1dc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e2de1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc06fd30 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e2de1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cd6a0 con 0x7fa1dc15bf40 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e2de1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc06f770 con 0x7fa1dc15ca30 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1dbfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa1dc0cccd0 0x7fa1dc15c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53298/0 (socket says 172.21.15.114:53298) 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1dbfff700 1 -- 172.21.15.114:0/2495756363 learned_addr learned my addr 172.21.15.114:0/2495756363 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e137e700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1dc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/2495756363 >> v2:172.21.15.172:3300/0 conn(0x7fa1dc15bf40 0x7fa1dc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213116329 0 0) 0x7fa1dc06fd30 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1cc0035b0 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846270455 0 0) 0x7fa1cc0035b0 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 msgr2=0x7fa1dc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1dc0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 >> v2:172.21.15.172:3300/0 conn(0x7fa1dc15bf40 msgr2=0x7fa1dc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 --2- 172.21.15.114:0/2495756363 >> v2:172.21.15.172:3300/0 conn(0x7fa1dc15bf40 0x7fa1dc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc06fa50 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fa1c8001670 con 0x7fa1dc0cccd0 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 >> v1:172.21.15.114:6789/0 conn(0x7fa1dc0cccd0 legacy=0x7fa1dc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1cc003a00 0x7fa1cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cd6a0 con 0x7fa1cc004850 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc06f770 con 0x7fa1cc003a00 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.882+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc00ba10 con 0x7fa1dc15ca30 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1cc003a00 0x7fa1cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1e137e700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 -- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 msgr2=0x7fa1cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc15ca30 0x7fa1cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 -- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1cc003a00 msgr2=0x7fa1cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1cc003a00 0x7fa1cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 -- 172.21.15.114:0/2495756363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0035b0 con 0x7fa1cc004850 2023-11-24T06:24:16.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1cc003a00 0x7fa1cc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1cc004cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c370 tx=0x7fa1cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643f13214107199c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8007c60 con 0x7fa1cc004850 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8005960 con 0x7fa1cc004850 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1db7fe700 1 -- 172.21.15.114:0/2495756363 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8005d10 con 0x7fa1cc004850 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.883+0000 7fa1e2de1700 1 -- 172.21.15.114:0/2495756363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 msgr2=0x7fa1cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/2495756363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1cc004cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c370 tx=0x7fa1cc005880 comp rx=0 tx=0).stop 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e137e700 1 -- 172.21.15.114:0/2495756363 reap_dead start 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e2de1700 1 -- 172.21.15.114:0/2495756363 shutdown_connections 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e2de1700 1 -- 172.21.15.114:0/2495756363 >> 172.21.15.114:0/2495756363 conn(0x7fa1dc0b2050 msgr2=0x7fa1dc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e2de1700 1 -- 172.21.15.114:0/2495756363 shutdown_connections 2023-11-24T06:24:16.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.884+0000 7fa1e2de1700 1 -- 172.21.15.114:0/2495756363 wait complete. 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.885+0000 7fa1e2de1700 1 Processor -- start 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.885+0000 7fa1e2de1700 1 -- start start 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 0x7fa1dc071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc0717a0 0x7fa1dc159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc071c30 con 0x7fa1cc004850 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc13f120 con 0x7fa1dc141e20 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc1551c0 con 0x7fa1dc0717a0 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37602/0 (socket says 172.21.15.114:37602) 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 -- 172.21.15.114:0/1089236114 learned_addr learned my addr 172.21.15.114:0/1089236114 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:16.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 0x7fa1dc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 -- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc0717a0 msgr2=0x7fa1dc159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e137e700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc0717a0 0x7fa1dc159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc0717a0 0x7fa1dc159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 -- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 msgr2=0x7fa1dc071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 0x7fa1dc071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 -- 172.21.15.114:0/1089236114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc1554a0 con 0x7fa1cc004850 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 0x7fa1dc071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e0b7d700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0010850 tx=0x7fa1d0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0010e50 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0004020 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc155780 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00043b0 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.886+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc155d40 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.887+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc068d90 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.888+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d000e8d0 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.888+0000 7fa1d97fa700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 0x7fa1c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.888+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa1d0089520 con 0x7fa1cc004850 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.888+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 0x7fa1c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:16.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.888+0000 7fa1dbfff700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 0x7fa1c006d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8015040 tx=0x7fa1c800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:16.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:16.893+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1d00549b0 con 0x7fa1cc004850 2023-11-24T06:24:17.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.083+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1dc0cfce0 con 0x7fa1cc004850 2023-11-24T06:24:17.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.083+0000 7fa1d97fa700 1 -- 172.21.15.114:0/1089236114 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1d0057fd0 con 0x7fa1cc004850 2023-11-24T06:24:17.086 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:17.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 msgr2=0x7fa1c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 0x7fa1c006d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8015040 tx=0x7fa1c800a000 comp rx=0 tx=0).stop 2023-11-24T06:24:17.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 msgr2=0x7fa1dc13ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0010850 tx=0x7fa1d0010c10 comp rx=0 tx=0).stop 2023-11-24T06:24:17.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 shutdown_connections 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa1c006aed0 0x7fa1c006d390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1dc0717a0 0x7fa1dc159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1cc004850 0x7fa1dc13ebe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 --2- 172.21.15.114:0/1089236114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1dc141e20 0x7fa1dc071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.085+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 >> 172.21.15.114:0/1089236114 conn(0x7fa1dc0b2050 msgr2=0x7fa1dc15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.086+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 shutdown_connections 2023-11-24T06:24:17.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.086+0000 7fa1e2de1700 1 -- 172.21.15.114:0/1089236114 wait complete. 2023-11-24T06:24:17.099 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:24:17.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.216+0000 7f49e1c4f700 1 Processor -- start 2023-11-24T06:24:17.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.216+0000 7f49e1c4f700 1 -- start start 2023-11-24T06:24:17.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc15ddd0 0x7f49dc0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f49dc15e1b0 0x7f49dc0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49dc0d1b30 0x7f49dc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0d2500 con 0x7f49dc15e1b0 2023-11-24T06:24:17.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0b1d40 con 0x7f49dc0d1b30 2023-11-24T06:24:17.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49e1c4f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0b28c0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f49dc15e1b0 0x7f49dc0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f49dc15e1b0 0x7f49dc0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37612/0 (socket says 172.21.15.114:37612) 2023-11-24T06:24:17.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 -- 172.21.15.114:0/3148370443 learned_addr learned my addr 172.21.15.114:0/3148370443 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:17.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49db7fe700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc15ddd0 0x7f49dc0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49dbfff700 1 --2- 172.21.15.114:0/3148370443 >> v2:172.21.15.172:3300/0 conn(0x7f49dc0d1b30 0x7f49dc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 -- 172.21.15.114:0/3148370443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc15ddd0 msgr2=0x7f49dc0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc15ddd0 0x7f49dc0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 -- 172.21.15.114:0/3148370443 >> v2:172.21.15.172:3300/0 conn(0x7f49dc0d1b30 msgr2=0x7f49dc0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 --2- 172.21.15.114:0/3148370443 >> v2:172.21.15.172:3300/0 conn(0x7f49dc0d1b30 0x7f49dc0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 -- 172.21.15.114:0/3148370443 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0b1f90 con 0x7f49dc15e1b0 2023-11-24T06:24:17.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49daffd700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15e1b0 0x7f49dc0cf000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0b2830 tx=0x7f49cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a6b36e1e048e4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49da7fc700 1 -- 172.21.15.114:0/3148370443 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc011830 con 0x7f49dc15e1b0 2023-11-24T06:24:17.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49da7fc700 1 -- 172.21.15.114:0/3148370443 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc011990 con 0x7f49dc15e1b0 2023-11-24T06:24:17.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.217+0000 7f49da7fc700 1 -- 172.21.15.114:0/3148370443 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc011d20 con 0x7f49dc15e1b0 2023-11-24T06:24:17.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 -- 172.21.15.114:0/3148370443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15e1b0 msgr2=0x7f49dc0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15e1b0 0x7f49dc0cf000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0b2830 tx=0x7f49cc009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 -- 172.21.15.114:0/3148370443 shutdown_connections 2023-11-24T06:24:17.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc15ddd0 0x7f49dc0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/3148370443 >> v2:172.21.15.172:3300/0 conn(0x7f49dc0d1b30 0x7f49dc0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/3148370443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15e1b0 0x7f49dc0cf000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 -- 172.21.15.114:0/3148370443 >> 172.21.15.114:0/3148370443 conn(0x7f49dc0731a0 msgr2=0x7f49dc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 -- 172.21.15.114:0/3148370443 shutdown_connections 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.218+0000 7f49e1c4f700 1 -- 172.21.15.114:0/3148370443 wait complete. 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.219+0000 7f49e1c4f700 1 Processor -- start 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.219+0000 7f49e1c4f700 1 -- start start 2023-11-24T06:24:17.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 0x7f49dc1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 0x7f49dc15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc09dc30 con 0x7f49dc15ddd0 2023-11-24T06:24:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc09cfb0 con 0x7f49dc15e1b0 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49e1c4f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc09d290 con 0x7f49dc0d1b30 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37616/0 (socket says 172.21.15.114:37616) 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 -- 172.21.15.114:0/1991773725 learned_addr learned my addr 172.21.15.114:0/1991773725 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49dbfff700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 0x7f49dc15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49db7fe700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 0x7f49dc1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 -- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 msgr2=0x7f49dc1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 0x7f49dc1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.220+0000 7f49daffd700 1 -- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 msgr2=0x7f49dc15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49daffd700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 0x7f49dc15fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49daffd700 1 -- 172.21.15.114:0/1991773725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc009550 con 0x7f49dc15ddd0 2023-11-24T06:24:17.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49db7fe700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 0x7f49dc1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:17.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49dbfff700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 0x7f49dc15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:17.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49daffd700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004300 con 0x7f49dc15ddd0 2023-11-24T06:24:17.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc01f070 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0b2ed0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0036a0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.221+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc09d850 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.222+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc09d9e0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.222+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49cc021030 con 0x7f49dc15ddd0 2023-11-24T06:24:17.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.222+0000 7f49c3fff700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 0x7f49c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.223+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49cc08a440 con 0x7f49dc15ddd0 2023-11-24T06:24:17.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.223+0000 7f49db7fe700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 0x7f49c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.223+0000 7f49db7fe700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 0x7f49c806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009c30 tx=0x7f49d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.227+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49cc0558d0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.420+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49dc067f60 con 0x7f49dc15ddd0 2023-11-24T06:24:17.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.421+0000 7f49c3fff700 1 -- 172.21.15.114:0/1991773725 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f49cc058ef0 con 0x7f49dc15ddd0 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:24:17.424 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 msgr2=0x7f49c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 0x7f49c806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009c30 tx=0x7f49d0008040 comp rx=0 tx=0).stop 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 msgr2=0x7f49dc15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc004750 comp rx=0 tx=0).stop 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 shutdown_connections 2023-11-24T06:24:17.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49c806aed0 0x7f49c806d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49dc15e1b0 0x7f49dc15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49dc0d1b30 0x7f49dc1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 --2- 172.21.15.114:0/1991773725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49dc15ddd0 0x7f49dc15f710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 >> 172.21.15.114:0/1991773725 conn(0x7f49dc0731a0 msgr2=0x7f49dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 shutdown_connections 2023-11-24T06:24:17.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.423+0000 7f49e1c4f700 1 -- 172.21.15.114:0/1991773725 wait complete. 2023-11-24T06:24:17.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:24:17.437 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:24:17.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.559+0000 7f828674f700 1 Processor -- start 2023-11-24T06:24:17.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.559+0000 7f828674f700 1 -- start start 2023-11-24T06:24:17.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f828674f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 0x7f82800726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f828674f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800ce160 con 0x7f82800af730 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f828674f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800cdc20 con 0x7f82800b1890 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f828674f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82800ce970 con 0x7f828015a4e0 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f8284cec700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f82800af730 0x7f82800afd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53336/0 (socket says 172.21.15.114:53336) 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f8284cec700 1 -- 172.21.15.114:0/2879105977 learned_addr learned my addr 172.21.15.114:0/2879105977 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827ffff700 1 --2- 172.21.15.114:0/2879105977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 0x7f82800726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048518361 0 0) 0x7f82800ce160 con 0x7f82800af730 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82680035b0 con 0x7f82800af730 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754226637 0 0) 0x7f82680035b0 con 0x7f82800af730 2023-11-24T06:24:17.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827ffff700 1 -- 172.21.15.114:0/2879105977 >> v1:172.21.15.172:6789/0 conn(0x7f82800b1890 legacy=0x7f8280072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827ffff700 1 -- 172.21.15.114:0/2879105977 >> v1:172.21.15.114:6789/0 conn(0x7f82800af730 legacy=0x7f82800afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827ffff700 1 -- 172.21.15.114:0/2879105977 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82800ce3b0 con 0x7f828015a4e0 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.560+0000 7f827ffff700 1 --2- 172.21.15.114:0/2879105977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 0x7f82800726c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f8274009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58bdf814dd14b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780019f0 con 0x7f828015a4e0 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274004030 con 0x7f828015a4e0 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f827effd700 1 -- 172.21.15.114:0/2879105977 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740043c0 con 0x7f828015a4e0 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 -- 172.21.15.114:0/2879105977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 msgr2=0x7f82800726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 --2- 172.21.15.114:0/2879105977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 0x7f82800726c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f8274009b80 comp rx=0 tx=0).stop 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 -- 172.21.15.114:0/2879105977 shutdown_connections 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 --2- 172.21.15.114:0/2879105977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828015a4e0 0x7f82800726c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 -- 172.21.15.114:0/2879105977 >> 172.21.15.114:0/2879105977 conn(0x7f82800b1e90 msgr2=0x7f82800b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 -- 172.21.15.114:0/2879105977 shutdown_connections 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.561+0000 7f828674f700 1 -- 172.21.15.114:0/2879105977 wait complete. 2023-11-24T06:24:17.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.562+0000 7f828674f700 1 Processor -- start 2023-11-24T06:24:17.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 -- start start 2023-11-24T06:24:17.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f828015a4e0 0x7f828013f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 0x7f8280143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280158c30 con 0x7f82800af730 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280156790 con 0x7f828015a4e0 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f828674f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8280159140 con 0x7f828013f560 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f827ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f827ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37630/0 (socket says 172.21.15.114:37630) 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.563+0000 7f827ffff700 1 -- 172.21.15.114:0/156921746 learned_addr learned my addr 172.21.15.114:0/156921746 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f8284cec700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 0x7f8280143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827f7fe700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f828015a4e0 0x7f828013f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 -- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 msgr2=0x7f8280143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 0x7f8280143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 -- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f828015a4e0 msgr2=0x7f828013f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f828015a4e0 0x7f828013f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 -- 172.21.15.114:0/156921746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740094a0 con 0x7f82800af730 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f8284cec700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 0x7f8280143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827ffff700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8280072400 tx=0x7f8274004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274010590 con 0x7f82800af730 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82740106f0 con 0x7f82800af730 2023-11-24T06:24:17.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8280158e60 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8280143930 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.564+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274010aa0 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.565+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8274010c00 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.565+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8280143ca0 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.566+0000 7f827cff9700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 0x7f826c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.566+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f827408ab20 con 0x7f82800af730 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.566+0000 7f827f7fe700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 0x7f826c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.566+0000 7f827f7fe700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 0x7f826c06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f828006f1e0 tx=0x7f8270009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.570+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8274055fb0 con 0x7f82800af730 2023-11-24T06:24:17.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.762+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8280143ac0 con 0x7f82800af730 2023-11-24T06:24:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.834+0000 7f827cff9700 1 -- 172.21.15.114:0/156921746 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f82740595d0 con 0x7f82800af730 2023-11-24T06:24:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 msgr2=0x7f826c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 0x7f826c06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f828006f1e0 tx=0x7f8270009040 comp rx=0 tx=0).stop 2023-11-24T06:24:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 msgr2=0x7f828006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8280072400 tx=0x7f8274004750 comp rx=0 tx=0).stop 2023-11-24T06:24:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 shutdown_connections 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f826c06aed0 0x7f826c06d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f828013f560 0x7f8280143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82800af730 0x7f828006e620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 --2- 172.21.15.114:0/156921746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f828015a4e0 0x7f828013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 >> 172.21.15.114:0/156921746 conn(0x7f82800b1e90 msgr2=0x7f82801505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 shutdown_connections 2023-11-24T06:24:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.836+0000 7f828674f700 1 -- 172.21.15.114:0/156921746 wait complete. 2023-11-24T06:24:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2023-11-24T06:24:17.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:24:17.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.980+0000 7f8411f55700 1 Processor -- start 2023-11-24T06:24:17.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.980+0000 7f8411f55700 1 -- start start 2023-11-24T06:24:17.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f8411f55700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f840c15bf50 0x7f840c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f8411f55700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f840c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f8411f55700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0d0140 con 0x7f840c0ce830 2023-11-24T06:24:17.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f8411f55700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15c330 con 0x7f840c15bf50 2023-11-24T06:24:17.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f8411f55700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0cf3c0 con 0x7f840c0c6c30 2023-11-24T06:24:17.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840affd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f840c0ce830 0x7f840c0cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53348/0 (socket says 172.21.15.114:53348) 2023-11-24T06:24:17.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840affd700 1 -- 172.21.15.114:0/3143164948 learned_addr learned my addr 172.21.15.114:0/3143164948 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840b7fe700 1 --2- 172.21.15.114:0/3143164948 >> v2:172.21.15.172:3300/0 conn(0x7f840c15bf50 0x7f840c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390823452 0 0) 0x7f840c0d0140 con 0x7f840c0ce830 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840bfff700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f840c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f80035b0 con 0x7f840c0ce830 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489861057 0 0) 0x7f83f80035b0 con 0x7f840c0ce830 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 msgr2=0x7f840c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f840c0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 >> v2:172.21.15.172:3300/0 conn(0x7f840c15bf50 msgr2=0x7f840c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 --2- 172.21.15.114:0/3143164948 >> v2:172.21.15.172:3300/0 conn(0x7f840c15bf50 0x7f840c0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0d03e0 con 0x7f840c0ce830 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.981+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f83f4001670 con 0x7f840c0ce830 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 >> v1:172.21.15.114:6789/0 conn(0x7f840c0ce830 legacy=0x7f840c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f83f8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83f8003f80 0x7f83f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f8004d90 0x7f83f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15c330 con 0x7f83f8003f80 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0cf3c0 con 0x7f83f8004d90 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f800ba10 con 0x7f840c0c6c30 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83f8003f80 0x7f83f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840b7fe700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f83f8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840bfff700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f8004d90 0x7f83f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 -- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 msgr2=0x7f83f8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f840c0c6c30 0x7f83f8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 -- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f8004d90 msgr2=0x7f83f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:17.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f8004d90 0x7f83f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 -- 172.21.15.114:0/3143164948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80035b0 con 0x7f83f8003f80 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840bfff700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f8004d90 0x7f83f8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840affd700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83f8003f80 0x7f83f8004850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c370 tx=0x7f83f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890595b2f73a36d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4007c60 con 0x7f83f8003f80 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f400ce50 con 0x7f83f8003f80 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.982+0000 7f840a7fc700 1 -- 172.21.15.114:0/3143164948 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4005ba0 con 0x7f83f8003f80 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 -- 172.21.15.114:0/3143164948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83f8003f80 msgr2=0x7f83f8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 --2- 172.21.15.114:0/3143164948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f83f8003f80 0x7f83f8004850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c370 tx=0x7f83f8005880 comp rx=0 tx=0).stop 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f840bfff700 1 -- 172.21.15.114:0/3143164948 reap_dead start 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 -- 172.21.15.114:0/3143164948 shutdown_connections 2023-11-24T06:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 -- 172.21.15.114:0/3143164948 >> 172.21.15.114:0/3143164948 conn(0x7f840c0b3780 msgr2=0x7f840c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 -- 172.21.15.114:0/3143164948 shutdown_connections 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.983+0000 7f8411f55700 1 -- 172.21.15.114:0/3143164948 wait complete. 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.984+0000 7f8411f55700 1 Processor -- start 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.984+0000 7f8411f55700 1 -- start start 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83f8003f80 0x7f840c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 0x7f840c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c1f4da0 con 0x7f840c1f19f0 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c154570 con 0x7f840c1561f0 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8411f55700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0b5ad0 con 0x7f83f8003f80 2023-11-24T06:24:17.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37652/0 (socket says 172.21.15.114:37652) 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 -- 172.21.15.114:0/4105119071 learned_addr learned my addr 172.21.15.114:0/4105119071 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840b7fe700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83f8003f80 0x7f840c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840bfff700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 0x7f840c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 -- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83f8003f80 msgr2=0x7f840c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83f8003f80 0x7f840c153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 -- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 msgr2=0x7f840c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 0x7f840c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 -- 172.21.15.114:0/4105119071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0b5db0 con 0x7f840c1f19f0 2023-11-24T06:24:17.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840bfff700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 0x7f840c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f840affd700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83f80046f0 tx=0x7f83f4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.985+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40070d0 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.986+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4007270 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.986+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c0b6090 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.986+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4007620 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.986+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c0b6650 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.987+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0b6810 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.987+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83f401e430 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.987+0000 7f8403fff700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 0x7f83ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.988+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f83f408a8e0 con 0x7f840c1f19f0 2023-11-24T06:24:17.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.988+0000 7f840b7fe700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 0x7f83ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:18.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.988+0000 7f840b7fe700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 0x7f83ec06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8404014510 tx=0x7f840400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:18.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:17.992+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83f4055d20 con 0x7f840c1f19f0 2023-11-24T06:24:18.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.185+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f840c0d0df0 con 0x7f840c1f19f0 2023-11-24T06:24:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.836+0000 7f8403fff700 1 -- 172.21.15.114:0/4105119071 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f83f4007dc0 con 0x7f840c1f19f0 2023-11-24T06:24:18.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 msgr2=0x7f83ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:18.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 0x7f83ec06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8404014510 tx=0x7f840400b040 comp rx=0 tx=0).stop 2023-11-24T06:24:18.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 msgr2=0x7f840c155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:18.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83f80046f0 tx=0x7f83f4000fe0 comp rx=0 tx=0).stop 2023-11-24T06:24:18.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 shutdown_connections 2023-11-24T06:24:18.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f83ec06aed0 0x7f83ec06d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:18.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f840c1561f0 0x7f840c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:18.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.839+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83f8003f80 0x7f840c153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:18.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.840+0000 7f8411f55700 1 --2- 172.21.15.114:0/4105119071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f840c1f19f0 0x7f840c155cb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:18.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.840+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 >> 172.21.15.114:0/4105119071 conn(0x7f840c0b3780 msgr2=0x7f840c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:18.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.840+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 shutdown_connections 2023-11-24T06:24:18.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.840+0000 7f8411f55700 1 -- 172.21.15.114:0/4105119071 wait complete. 2023-11-24T06:24:18.850 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2023-11-24T06:24:18.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:24:18.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.994+0000 7efee0c2e700 1 Processor -- start 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.994+0000 7efee0c2e700 1 -- start start 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.995+0000 7efee0c2e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.995+0000 7efee0c2e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.995+0000 7efee0c2e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc0b2f20 con 0x7efedc06eb20 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.995+0000 7efee0c2e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc06efb0 con 0x7efedc0cc540 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.995+0000 7efee0c2e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0ce690 con 0x7efedc15a620 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed9d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60434/0 (socket says 172.21.15.114:60434) 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed9d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33316/0 (socket says 172.21.15.114:33316) 2023-11-24T06:24:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 -- 172.21.15.114:0/3028784761 learned_addr learned my addr 172.21.15.114:0/3028784761 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 -- 172.21.15.114:0/3028784761 >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 msgr2=0x7efedc06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 --2- 172.21.15.114:0/3028784761 >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 -- 172.21.15.114:0/3028784761 >> v1:172.21.15.172:6789/0 conn(0x7efedc06eb20 legacy=0x7efedc06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 -- 172.21.15.114:0/3028784761 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc15aa00 con 0x7efedc15a620 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed9d9b700 1 --2- 172.21.15.114:0/3028784761 >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efeda59c700 1 --2- 172.21.15.114:0/3028784761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efed000aa90 tx=0x7efed000ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3599a9c6a04c6128 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed959a700 1 -- 172.21.15.114:0/3028784761 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000cb70 con 0x7efedc15a620 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed959a700 1 -- 172.21.15.114:0/3028784761 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000ccd0 con 0x7efedc15a620 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.996+0000 7efed959a700 1 -- 172.21.15.114:0/3028784761 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0018620 con 0x7efedc15a620 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 -- 172.21.15.114:0/3028784761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 msgr2=0x7efedc0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 --2- 172.21.15.114:0/3028784761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efed000aa90 tx=0x7efed000ada0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 -- 172.21.15.114:0/3028784761 shutdown_connections 2023-11-24T06:24:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 --2- 172.21.15.114:0/3028784761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc15a620 0x7efedc0ce150 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 --2- 172.21.15.114:0/3028784761 >> v2:172.21.15.172:3300/0 conn(0x7efedc0cc540 0x7efedc06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 -- 172.21.15.114:0/3028784761 >> 172.21.15.114:0/3028784761 conn(0x7efedc0b2050 msgr2=0x7efedc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 -- 172.21.15.114:0/3028784761 shutdown_connections 2023-11-24T06:24:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.997+0000 7efee0c2e700 1 -- 172.21.15.114:0/3028784761 wait complete. 2023-11-24T06:24:19.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.998+0000 7efee0c2e700 1 Processor -- start 2023-11-24T06:24:19.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.998+0000 7efee0c2e700 1 -- start start 2023-11-24T06:24:19.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 0x7efedc147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc0cc540 0x7efedc150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc148460 con 0x7efedc15a620 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc150ab0 con 0x7efedc06eb20 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efee0c2e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc150d30 con 0x7efedc0cc540 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efedad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efedad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37666/0 (socket says 172.21.15.114:37666) 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efedad9d700 1 -- 172.21.15.114:0/4140339342 learned_addr learned my addr 172.21.15.114:0/4140339342 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:18.999+0000 7efed9d9b700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc0cc540 0x7efedc150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efeda59c700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 0x7efedc147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 -- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc0cc540 msgr2=0x7efedc150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc0cc540 0x7efedc150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 -- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 msgr2=0x7efedc147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 0x7efedc147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 -- 172.21.15.114:0/4140339342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0094a0 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efeda59c700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 0x7efedc147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efedad9d700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efed400cc50 tx=0x7efed400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4003c00 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed4003d60 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed000a6f0 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400ab60 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.000+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc1512f0 con 0x7efedc15a620 2023-11-24T06:24:19.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.001+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc068d90 con 0x7efedc15a620 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.001+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efed400acc0 con 0x7efedc15a620 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.002+0000 7efecaffd700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 0x7efebc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.002+0000 7efeda59c700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 0x7efebc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.002+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7efed4089670 con 0x7efedc15a620 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.002+0000 7efeda59c700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 0x7efebc06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efed000ade0 tx=0x7efed001a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:19.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.006+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efed40546f0 con 0x7efedc15a620 2023-11-24T06:24:19.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.242+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efedc068fb0 con 0x7efedc15a620 2023-11-24T06:24:19.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.864+0000 7efecaffd700 1 -- 172.21.15.114:0/4140339342 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7efed4057d10 con 0x7efedc15a620 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 msgr2=0x7efebc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 0x7efebc06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efed000ade0 tx=0x7efed001a040 comp rx=0 tx=0).stop 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 msgr2=0x7efedc0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efed400cc50 tx=0x7efed400cf60 comp rx=0 tx=0).stop 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 shutdown_connections 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.881+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efebc06aed0 0x7efebc06d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efedc15a620 0x7efedc0b1010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efedc06eb20 0x7efedc147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 --2- 172.21.15.114:0/4140339342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efedc0cc540 0x7efedc150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 >> 172.21.15.114:0/4140339342 conn(0x7efedc0b2050 msgr2=0x7efedc071260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 shutdown_connections 2023-11-24T06:24:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:19.882+0000 7efee0c2e700 1 -- 172.21.15.114:0/4140339342 wait complete. 2023-11-24T06:24:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:24:19.900 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:20.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.022+0000 7f38c7f87700 1 Processor -- start 2023-11-24T06:24:20.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 -- start start 2023-11-24T06:24:20.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 0x7f38c00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38c006c710 0x7f38c00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00ae430 con 0x7f38c00af4d0 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c00af0b0 con 0x7f38c006c710 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.023+0000 7f38c7f87700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b05a0 con 0x7f38c015e080 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38c006c710 0x7f38c00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c5522700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f38c00af4d0 0x7f38c006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53396/0 (socket says 172.21.15.114:53396) 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c5522700 1 -- 172.21.15.114:0/2978056327 learned_addr learned my addr 172.21.15.114:0/2978056327 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c5d23700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 0x7f38c00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 -- 172.21.15.114:0/2978056327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 msgr2=0x7f38c00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 0x7f38c00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 -- 172.21.15.114:0/2978056327 >> v1:172.21.15.114:6789/0 conn(0x7f38c00af4d0 legacy=0x7f38c006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 -- 172.21.15.114:0/2978056327 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00ae710 con 0x7f38c006c710 2023-11-24T06:24:20.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c4d21700 1 -- 172.21.15.114:0/2978056327 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929361965 0 0) 0x7f38c00ae430 con 0x7f38c00af4d0 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c5d23700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 0x7f38c00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c6524700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c006c710 0x7f38c00afb90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009dc0 tx=0x7f38b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549f72288d83169b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c4d21700 1 -- 172.21.15.114:0/2978056327 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b800e070 con 0x7f38c006c710 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c4d21700 1 -- 172.21.15.114:0/2978056327 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b800bb90 con 0x7f38c006c710 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.024+0000 7f38c4d21700 1 -- 172.21.15.114:0/2978056327 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012430 con 0x7f38c006c710 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 -- 172.21.15.114:0/2978056327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c006c710 msgr2=0x7f38c00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c006c710 0x7f38c00afb90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009dc0 tx=0x7f38b800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 -- 172.21.15.114:0/2978056327 shutdown_connections 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c015e080 0x7f38c00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 --2- 172.21.15.114:0/2978056327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c006c710 0x7f38c00afb90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 -- 172.21.15.114:0/2978056327 >> 172.21.15.114:0/2978056327 conn(0x7f38c0071b90 msgr2=0x7f38c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 -- 172.21.15.114:0/2978056327 shutdown_connections 2023-11-24T06:24:20.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.025+0000 7f38c7f87700 1 -- 172.21.15.114:0/2978056327 wait complete. 2023-11-24T06:24:20.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.026+0000 7f38c7f87700 1 Processor -- start 2023-11-24T06:24:20.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.026+0000 7f38c7f87700 1 -- start start 2023-11-24T06:24:20.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 0x7f38c015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c00af4d0 0x7f38c0152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0157400 con 0x7f38c015e080 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01576e0 con 0x7f38c006c710 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01579c0 con 0x7f38c00af4d0 2023-11-24T06:24:20.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37694/0 (socket says 172.21.15.114:37694) 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 -- 172.21.15.114:0/833453611 learned_addr learned my addr 172.21.15.114:0/833453611 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c5522700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c00af4d0 0x7f38c0152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c5d23700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 0x7f38c015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 -- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c00af4d0 msgr2=0x7f38c0152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c00af4d0 0x7f38c0152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 -- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 msgr2=0x7f38c015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 0x7f38c015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 -- 172.21.15.114:0/833453611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8009a70 con 0x7f38c015e080 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c5d23700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 0x7f38c015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c6524700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38b8014040 tx=0x7f38b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b801b810 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 -- 172.21.15.114:0/833453611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c0157ca0 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8020070 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38c7f87700 1 -- 172.21.15.114:0/833453611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c0158260 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.027+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b801bb70 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.028+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c0052e0 con 0x7f38c015e080 2023-11-24T06:24:20.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.029+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38b801bcd0 con 0x7f38c015e080 2023-11-24T06:24:20.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.029+0000 7f38b67fc700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 0x7f38a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.029+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38b8022070 con 0x7f38c015e080 2023-11-24T06:24:20.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.029+0000 7f38c5d23700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 0x7f38a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.030+0000 7f38c5d23700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 0x7f38a406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009960 tx=0x7f38b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.034+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38b805aa30 con 0x7f38c015e080 2023-11-24T06:24:20.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.224+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f389c005590 con 0x7f38c015e080 2023-11-24T06:24:20.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.225+0000 7f38b67fc700 1 -- 172.21.15.114:0/833453611 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f38b805a5c0 con 0x7f38c015e080 2023-11-24T06:24:20.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:20.228 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:19.243297+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 msgr2=0x7f38a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 0x7f38a406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009960 tx=0x7f38b0008040 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 msgr2=0x7f38c0152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38b8014040 tx=0x7f38b8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 shutdown_connections 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38a406aed0 0x7f38a406d390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38c015e080 0x7f38c0152d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c006c710 0x7f38c015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 --2- 172.21.15.114:0/833453611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38c00af4d0 0x7f38c0152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 >> 172.21.15.114:0/833453611 conn(0x7f38c0071b90 msgr2=0x7f38c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 shutdown_connections 2023-11-24T06:24:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.227+0000 7f38a3fff700 1 -- 172.21.15.114:0/833453611 wait complete. 2023-11-24T06:24:20.242 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:20.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.360+0000 7f6760ddc700 1 Processor -- start 2023-11-24T06:24:20.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.360+0000 7f6760ddc700 1 -- start start 2023-11-24T06:24:20.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6760ddc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f675c0af4d0 0x7f675c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6760ddc700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0ae430 con 0x7f675c06c710 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6760ddc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0ae710 con 0x7f675c15e080 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6760ddc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c0af0b0 con 0x7f675c0af4d0 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6759d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f675c06c710 0x7f675c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53408/0 (socket says 172.21.15.114:53408) 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f6759d9b700 1 -- 172.21.15.114:0/418611777 learned_addr learned my addr 172.21.15.114:0/418611777 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675ad9d700 1 --2- 172.21.15.114:0/418611777 >> v2:172.21.15.114:3300/0 conn(0x7f675c0af4d0 0x7f675c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675959a700 1 -- 172.21.15.114:0/418611777 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379595356 0 0) 0x7f675c0ae430 con 0x7f675c06c710 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675959a700 1 -- 172.21.15.114:0/418611777 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67480035b0 con 0x7f675c06c710 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675ad9d700 1 -- 172.21.15.114:0/418611777 >> v1:172.21.15.172:6789/0 conn(0x7f675c15e080 legacy=0x7f675c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675ad9d700 1 -- 172.21.15.114:0/418611777 >> v1:172.21.15.114:6789/0 conn(0x7f675c06c710 legacy=0x7f675c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675ad9d700 1 -- 172.21.15.114:0/418611777 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0ae9f0 con 0x7f675c0af4d0 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675ad9d700 1 --2- 172.21.15.114:0/418611777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c0af4d0 0x7f675c0b5050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f674c009d90 tx=0x7f674c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a2d5067c14a5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675959a700 1 -- 172.21.15.114:0/418611777 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00e070 con 0x7f675c0af4d0 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675959a700 1 -- 172.21.15.114:0/418611777 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c00bb20 con 0x7f675c0af4d0 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.361+0000 7f675959a700 1 -- 172.21.15.114:0/418611777 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c012430 con 0x7f675c0af4d0 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 -- 172.21.15.114:0/418611777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c0af4d0 msgr2=0x7f675c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 --2- 172.21.15.114:0/418611777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c0af4d0 0x7f675c0b5050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f674c009d90 tx=0x7f674c00b330 comp rx=0 tx=0).stop 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 -- 172.21.15.114:0/418611777 shutdown_connections 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 --2- 172.21.15.114:0/418611777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c0af4d0 0x7f675c0b5050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 -- 172.21.15.114:0/418611777 >> 172.21.15.114:0/418611777 conn(0x7f675c071b90 msgr2=0x7f675c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 -- 172.21.15.114:0/418611777 shutdown_connections 2023-11-24T06:24:20.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.362+0000 7f6760ddc700 1 -- 172.21.15.114:0/418611777 wait complete. 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.363+0000 7f6760ddc700 1 Processor -- start 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.363+0000 7f6760ddc700 1 -- start start 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f675c0af4d0 0x7f675c160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f675c15e080 0x7f675c142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c15b7c0 con 0x7f675c06c710 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0cf870 con 0x7f675c0af4d0 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6760ddc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c161410 con 0x7f675c15e080 2023-11-24T06:24:20.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37704/0 (socket says 172.21.15.114:37704) 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 -- 172.21.15.114:0/2268270789 learned_addr learned my addr 172.21.15.114:0/2268270789 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f6759d9b700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f675c0af4d0 0x7f675c160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675ad9d700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f675c15e080 0x7f675c142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 -- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f675c15e080 msgr2=0x7f675c142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f675c15e080 0x7f675c142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 -- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f675c0af4d0 msgr2=0x7f675c160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f675c0af4d0 0x7f675c160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.364+0000 7f675a59c700 1 -- 172.21.15.114:0/2268270789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c009a70 con 0x7f675c06c710 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f675a59c700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f675400c480 tx=0x7f675400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540052d0 con 0x7f675c06c710 2023-11-24T06:24:20.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6754005430 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c0cef80 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540038c0 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.365+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c0cf220 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.366+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c0753f0 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.366+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6754003a20 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.366+0000 7f6752ffd700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 0x7f673c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.366+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67540888d0 con 0x7f675c06c710 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.366+0000 7f6759d9b700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 0x7f673c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.367+0000 7f6759d9b700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 0x7f673c06d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f675c15e480 tx=0x7f6744007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.371+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6754053980 con 0x7f675c06c710 2023-11-24T06:24:20.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.562+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f675c067f60 con 0x7f675c06c710 2023-11-24T06:24:20.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.563+0000 7f6752ffd700 1 -- 172.21.15.114:0/2268270789 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f6754017090 con 0x7f675c06c710 2023-11-24T06:24:20.568 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:20.569 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:19.243297+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:20.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 msgr2=0x7f673c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 0x7f673c06d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f675c15e480 tx=0x7f6744007dc0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 msgr2=0x7f675c141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f675400c480 tx=0x7f675400c790 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 shutdown_connections 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f673c06aed0 0x7f673c06d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f675c15e080 0x7f675c142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f675c06c710 0x7f675c141e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 --2- 172.21.15.114:0/2268270789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f675c0af4d0 0x7f675c160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.565+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 >> 172.21.15.114:0/2268270789 conn(0x7f675c071b90 msgr2=0x7f675c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.566+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 shutdown_connections 2023-11-24T06:24:20.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.566+0000 7f6760ddc700 1 -- 172.21.15.114:0/2268270789 wait complete. 2023-11-24T06:24:20.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:24:20.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.701+0000 7fc5ab22f700 1 Processor -- start 2023-11-24T06:24:20.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.701+0000 7fc5ab22f700 1 -- start start 2023-11-24T06:24:20.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5a415e160 0x7fc5a40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40afc90 0x7fc5a4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc5a40744d0 0x7fc5a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40b4500 con 0x7fc5a415e160 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a4074960 con 0x7fc5a40744d0 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5ab22f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b3c60 con 0x7fc5a40afc90 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5a415e160 0x7fc5a40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5a415e160 0x7fc5a40b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37712/0 (socket says 172.21.15.114:37712) 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 -- 172.21.15.114:0/3116533234 learned_addr learned my addr 172.21.15.114:0/3116533234 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a3fff700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40afc90 0x7fc5a4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a97cc700 1 --2- 172.21.15.114:0/3116533234 >> v2:172.21.15.172:3300/0 conn(0x7fc5a40744d0 0x7fc5a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 -- 172.21.15.114:0/3116533234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40afc90 msgr2=0x7fc5a4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:20.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40afc90 0x7fc5a4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 -- 172.21.15.114:0/3116533234 >> v2:172.21.15.172:3300/0 conn(0x7fc5a40744d0 msgr2=0x7fc5a406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/3116533234 >> v2:172.21.15.172:3300/0 conn(0x7fc5a40744d0 0x7fc5a406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.702+0000 7fc5a8fcb700 1 -- 172.21.15.114:0/3116533234 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40b58c0 con 0x7fc5a415e160 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a40b5da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40d1cd0 tx=0x7fc598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2895eebc74aa67c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5a37fe700 1 -- 172.21.15.114:0/3116533234 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011830 con 0x7fc5a415e160 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5a37fe700 1 -- 172.21.15.114:0/3116533234 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598011990 con 0x7fc5a415e160 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5a37fe700 1 -- 172.21.15.114:0/3116533234 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011d20 con 0x7fc5a415e160 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 -- 172.21.15.114:0/3116533234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 msgr2=0x7fc5a40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a40b5da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40d1cd0 tx=0x7fc598009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 -- 172.21.15.114:0/3116533234 shutdown_connections 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/3116533234 >> v2:172.21.15.172:3300/0 conn(0x7fc5a40744d0 0x7fc5a406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a40b5da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/3116533234 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40afc90 0x7fc5a4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 -- 172.21.15.114:0/3116533234 >> 172.21.15.114:0/3116533234 conn(0x7fc5a40d0bf0 msgr2=0x7fc5a40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 -- 172.21.15.114:0/3116533234 shutdown_connections 2023-11-24T06:24:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.703+0000 7fc5ab22f700 1 -- 172.21.15.114:0/3116533234 wait complete. 2023-11-24T06:24:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.705+0000 7fc5ab22f700 1 Processor -- start 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.705+0000 7fc5ab22f700 1 -- start start 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40744d0 0x7fc5a4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 0x7fc5a415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41f4f30 con 0x7fc5a415e160 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41f4970 con 0x7fc5a40afc90 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5ab22f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41f4c50 con 0x7fc5a40744d0 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37722/0 (socket says 172.21.15.114:37722) 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 -- 172.21.15.114:0/303402765 learned_addr learned my addr 172.21.15.114:0/303402765 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40744d0 0x7fc5a4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a3fff700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 0x7fc5a415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 -- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40744d0 msgr2=0x7fc5a4154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40744d0 0x7fc5a4154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 -- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 msgr2=0x7fc5a415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 0x7fc5a415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a97cc700 1 -- 172.21.15.114:0/303402765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009550 con 0x7fc5a415e160 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.706+0000 7fc5a3fff700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 0x7fc5a415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5a97cc700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00efc0 tx=0x7fc59c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009ea0 con 0x7fc5a415e160 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00f040 con 0x7fc5a415e160 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a4156030 con 0x7fc5a415e160 2023-11-24T06:24:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c014480 con 0x7fc5a415e160 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.707+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a415ead0 con 0x7fc5a415e160 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.708+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a415ecf0 con 0x7fc5a415e160 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.708+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc59c0145e0 con 0x7fc5a415e160 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.708+0000 7fc5a17fa700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 0x7fc58806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.709+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 0x7fc58806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.709+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc59c089ed0 con 0x7fc5a415e160 2023-11-24T06:24:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.709+0000 7fc5a8fcb700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 0x7fc58806d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc598014040 tx=0x7fc598008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.713+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc59c054f80 con 0x7fc5a415e160 2023-11-24T06:24:20.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:20.952+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5a4068fb0 con 0x7fc5a415e160 2023-11-24T06:24:21.285 INFO:tasks.ceph.mds.c.smithi114.stderr:2023-11-24T06:24:21.283+0000 7fb808c0f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:24:21.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.872+0000 7fc5a17fa700 1 -- 172.21.15.114:0/303402765 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc59c0180b0 con 0x7fc5a415e160 2023-11-24T06:24:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 msgr2=0x7fc58806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 0x7fc58806d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc598014040 tx=0x7fc598008030 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 msgr2=0x7fc5a41f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00efc0 tx=0x7fc59c00c3a0 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 shutdown_connections 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc58806aed0 0x7fc58806d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5a415e160 0x7fc5a41f3f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5a40744d0 0x7fc5a4154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 --2- 172.21.15.114:0/303402765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5a40afc90 0x7fc5a415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 >> 172.21.15.114:0/303402765 conn(0x7fc5a40d0bf0 msgr2=0x7fc5a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 shutdown_connections 2023-11-24T06:24:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:21.875+0000 7fc5ab22f700 1 -- 172.21.15.114:0/303402765 wait complete. 2023-11-24T06:24:21.892 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:24:21.893 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:24:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.018+0000 7f7bffd01700 1 Processor -- start 2023-11-24T06:24:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.018+0000 7f7bffd01700 1 -- start start 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bffd01700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7bf80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bffd01700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80b3c60 con 0x7f7bf80746a0 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bffd01700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80b58c0 con 0x7f7bf80afc90 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bffd01700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b4500 con 0x7f7bf815e160 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfe29e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7bf80746a0 0x7f7bf806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53438/0 (socket says 172.21.15.114:53438) 2023-11-24T06:24:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfe29e700 1 -- 172.21.15.114:0/2476189692 learned_addr learned my addr 172.21.15.114:0/2476189692 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfda9d700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7bf80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80396060 0 0) 0x7f7bf80b3c60 con 0x7f7bf80746a0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be40035b0 con 0x7f7bf80746a0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471965333 0 0) 0x7f7bf80b58c0 con 0x7f7bf80afc90 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf80b3c60 con 0x7f7bf80afc90 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4256489679 0 0) 0x7f7be40035b0 con 0x7f7bf80746a0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 msgr2=0x7f7bf80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7bf80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 >> v1:172.21.15.172:6789/0 conn(0x7f7bf80afc90 legacy=0x7f7bf8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf806bdc0 con 0x7f7bf80746a0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f7bf00019f0 con 0x7f7bf80746a0 2023-11-24T06:24:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 >> v1:172.21.15.114:6789/0 conn(0x7f7bf80746a0 legacy=0x7f7bf806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7be4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7be4004170 0x7f7be4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7be4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.019+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b4500 con 0x7f7be4004d90 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be400bb50 con 0x7f7be4004170 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be400bd10 con 0x7f7bf815e160 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7be4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfda9d700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7be4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfd29c700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7be4004170 0x7f7be4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 -- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 msgr2=0x7f7be4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815e160 0x7f7be4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 -- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7be4004170 msgr2=0x7f7be4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7be4004170 0x7f7be4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 -- 172.21.15.114:0/2476189692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40035b0 con 0x7f7be4004d90 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfe29e700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7be4005210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000a6e0 tx=0x7f7be4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4223a5c75c5f062 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00072d0 con 0x7f7be4004d90 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0007430 con 0x7f7be4004d90 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.020+0000 7f7bfca9b700 1 -- 172.21.15.114:0/2476189692 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00077e0 con 0x7f7be4004d90 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 -- 172.21.15.114:0/2476189692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 msgr2=0x7f7be4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 --2- 172.21.15.114:0/2476189692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7be4005210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000a6e0 tx=0x7f7be4005880 comp rx=0 tx=0).stop 2023-11-24T06:24:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bfe29e700 1 -- 172.21.15.114:0/2476189692 reap_dead start 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 -- 172.21.15.114:0/2476189692 shutdown_connections 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 -- 172.21.15.114:0/2476189692 >> 172.21.15.114:0/2476189692 conn(0x7f7bf80d0bf0 msgr2=0x7f7bf80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 -- 172.21.15.114:0/2476189692 shutdown_connections 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.021+0000 7f7bffd01700 1 -- 172.21.15.114:0/2476189692 wait complete. 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.022+0000 7f7bffd01700 1 Processor -- start 2023-11-24T06:24:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.022+0000 7f7bffd01700 1 -- start start 2023-11-24T06:24:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 0x7f7bf81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf8158070 0x7f7bf814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815f980 con 0x7f7be4004d90 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8157630 con 0x7f7bf8158070 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815ed00 con 0x7f7bf815fda0 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37738/0 (socket says 172.21.15.114:37738) 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 -- 172.21.15.114:0/4116787299 learned_addr learned my addr 172.21.15.114:0/4116787299 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfe29e700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf8158070 0x7f7bf814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfd29c700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 0x7f7bf81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 -- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 msgr2=0x7f7bf81570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 0x7f7bf81570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 -- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf8158070 msgr2=0x7f7bf814b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf8158070 0x7f7bf814b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 -- 172.21.15.114:0/4116787299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf815efe0 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfd29c700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 0x7f7bf81570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bfda9d700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4011aa0 tx=0x7f7bf4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400f940 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400faa0 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf815f2c0 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400fe50 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.023+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81584e0 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.024+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8068fb0 con 0x7f7be4004d90 2023-11-24T06:24:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.025+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bf4014440 con 0x7f7be4004d90 2023-11-24T06:24:22.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.025+0000 7f7bee7fc700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 0x7f7bdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:22.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.025+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bf4089df0 con 0x7f7be4004d90 2023-11-24T06:24:22.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.025+0000 7f7bfd29c700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 0x7f7bdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:22.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.025+0000 7f7bfd29c700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 0x7f7bdc06d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7be8007fd0 tx=0x7f7be8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:22.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.030+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bf4054f80 con 0x7f7be4004d90 2023-11-24T06:24:22.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.263+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7bf806c140 con 0x7f7be4004d90 2023-11-24T06:24:22.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.873+0000 7f7bee7fc700 1 -- 172.21.15.114:0/4116787299 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7bf4022070 con 0x7f7be4004d90 2023-11-24T06:24:22.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 msgr2=0x7f7bdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 0x7f7bdc06d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7be8007fd0 tx=0x7f7be8009040 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 msgr2=0x7f7bf815e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4011aa0 tx=0x7f7bf4011db0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 shutdown_connections 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7bdc06aed0 0x7f7bdc06d390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bf8158070 0x7f7bf814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7be4004d90 0x7f7bf815e720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 --2- 172.21.15.114:0/4116787299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bf815fda0 0x7f7bf81570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 >> 172.21.15.114:0/4116787299 conn(0x7f7bf80d0bf0 msgr2=0x7f7bf80af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 shutdown_connections 2023-11-24T06:24:22.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:22.876+0000 7f7bffd01700 1 -- 172.21.15.114:0/4116787299 wait complete. 2023-11-24T06:24:22.893 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:24:22.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:23.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.014+0000 7efed1185700 1 Processor -- start 2023-11-24T06:24:23.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.014+0000 7efed1185700 1 -- start start 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efed1185700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efecc15ae20 0x7efecc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efed1185700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0cfd40 con 0x7efecc0b1a70 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efed1185700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0affa0 con 0x7efecc15b200 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efed1185700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0d0020 con 0x7efecc15ae20 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efecb59e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7efecc0b1a70 0x7efecc164b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53458/0 (socket says 172.21.15.114:53458) 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efecb59e700 1 -- 172.21.15.114:0/1922278585 learned_addr learned my addr 172.21.15.114:0/1922278585 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efecad9d700 1 --2- 172.21.15.114:0/1922278585 >> v2:172.21.15.172:3300/0 conn(0x7efecc15ae20 0x7efecc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178039343 0 0) 0x7efecc0cfd40 con 0x7efecc0b1a70 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb40035b0 con 0x7efecc0b1a70 2023-11-24T06:24:23.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464714623 0 0) 0x7efeb40035b0 con 0x7efecc0b1a70 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 >> v2:172.21.15.172:3300/0 conn(0x7efecc15ae20 msgr2=0x7efecc0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 --2- 172.21.15.114:0/1922278585 >> v2:172.21.15.172:3300/0 conn(0x7efecc15ae20 0x7efecc0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 >> v1:172.21.15.172:6789/0 conn(0x7efecc15b200 legacy=0x7efecc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0b0280 con 0x7efecc0b1a70 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7efec40019f0 con 0x7efecc0b1a70 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 >> v1:172.21.15.114:6789/0 conn(0x7efecc0b1a70 legacy=0x7efecc164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efeb4003a00 0x7efeb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efeb4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efeb4005210 0x7efeb4010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0d0020 con 0x7efeb4004850 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40105c0 con 0x7efeb4003a00 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb400bc10 con 0x7efeb4005210 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efeca59c700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efeb4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efecad9d700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efeb4003a00 0x7efeb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.015+0000 7efecb59e700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efeb4005210 0x7efeb4010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 -- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efeb4005210 msgr2=0x7efeb4010020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efeb4005210 0x7efeb4010020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 -- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efeb4003a00 msgr2=0x7efeb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efeb4003a00 0x7efeb4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 -- 172.21.15.114:0/1922278585 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40035b0 con 0x7efeb4004850 2023-11-24T06:24:23.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efecb59e700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efeb4005210 0x7efeb4010020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:23.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efecb59e700 1 -- 172.21.15.114:0/1922278585 reap_dead start 2023-11-24T06:24:23.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efeca59c700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efeb4004cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efecc0717c0 tx=0x7efeb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c2632f16f588dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00052d0 con 0x7efeb4004850 2023-11-24T06:24:23.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0005430 con 0x7efeb4004850 2023-11-24T06:24:23.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efec9d9b700 1 -- 172.21.15.114:0/1922278585 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00038c0 con 0x7efeb4004850 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 -- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 msgr2=0x7efeb4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efeb4004cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efecc0717c0 tx=0x7efeb4005880 comp rx=0 tx=0).stop 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 -- 172.21.15.114:0/1922278585 shutdown_connections 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 --2- 172.21.15.114:0/1922278585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efeb4004cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 -- 172.21.15.114:0/1922278585 >> 172.21.15.114:0/1922278585 conn(0x7efecc0b57b0 msgr2=0x7efecc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 -- 172.21.15.114:0/1922278585 shutdown_connections 2023-11-24T06:24:23.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.016+0000 7efed1185700 1 -- 172.21.15.114:0/1922278585 wait complete. 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.018+0000 7efed1185700 1 Processor -- start 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.018+0000 7efed1185700 1 -- start start 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efecc15c970 0x7efecc1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efecc15e150 0x7efecc0ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15e910 con 0x7efeb4004850 2023-11-24T06:24:23.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15c4e0 con 0x7efecc15e150 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1f3c10 con 0x7efecc15c970 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37760/0 (socket says 172.21.15.114:37760) 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 -- 172.21.15.114:0/2172146095 learned_addr learned my addr 172.21.15.114:0/2172146095 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecb59e700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efecc15e150 0x7efecc0ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efeca59c700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efecc15c970 0x7efecc1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 -- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efecc15c970 msgr2=0x7efecc1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efecc15c970 0x7efecc1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 -- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efecc15e150 msgr2=0x7efecc0ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efecc15e150 0x7efecc0ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 -- 172.21.15.114:0/2172146095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc15bbb0 con 0x7efeb4004850 2023-11-24T06:24:23.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efecad9d700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efebc011ad0 tx=0x7efebc011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00f8e0 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00fa40 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc15be90 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00fdf0 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.019+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc15e5d0 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.020+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc15c7c0 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.021+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efebc014440 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.021+0000 7efeb37fe700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 0x7efeb806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.021+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efebc089c10 con 0x7efeb4004850 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.021+0000 7efeca59c700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 0x7efeb806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.021+0000 7efeca59c700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 0x7efeb806d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efec0000c00 tx=0x7efec0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.026+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efebc054cc0 con 0x7efeb4004850 2023-11-24T06:24:23.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.258+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efecc067f60 con 0x7efeb4004850 2023-11-24T06:24:23.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.259+0000 7efeb37fe700 1 -- 172.21.15.114:0/2172146095 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7efebc0582e0 con 0x7efeb4004850 2023-11-24T06:24:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:23.262 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:24:23.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 msgr2=0x7efeb806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 0x7efeb806d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efec0000c00 tx=0x7efec0009030 comp rx=0 tx=0).stop 2023-11-24T06:24:23.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 msgr2=0x7efecc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efebc011ad0 tx=0x7efebc011de0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 shutdown_connections 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efeb806aed0 0x7efeb806d390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efecc15e150 0x7efecc0ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efeb4004850 0x7efecc0cd610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 --2- 172.21.15.114:0/2172146095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efecc15c970 0x7efecc1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 >> 172.21.15.114:0/2172146095 conn(0x7efecc0b57b0 msgr2=0x7efecc0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 shutdown_connections 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.261+0000 7efed1185700 1 -- 172.21.15.114:0/2172146095 wait complete. 2023-11-24T06:24:23.265 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:24:23.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.393+0000 7f7a1397f700 1 Processor -- start 2023-11-24T06:24:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.393+0000 7f7a1397f700 1 -- start start 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a1397f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7a0c0af4d0 0x7f7a0c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a1397f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0ae430 con 0x7f7a0c06c710 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a1397f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0ae710 con 0x7f7a0c15e080 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a1397f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0af0b0 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a11f1c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7a0c0af4d0 0x7f7a0c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a10f1a700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7a0c06c710 0x7f7a0c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53476/0 (socket says 172.21.15.114:53476) 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a10f1a700 1 -- 172.21.15.114:0/870236370 learned_addr learned my addr 172.21.15.114:0/870236370 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a11f1c700 1 -- 172.21.15.114:0/870236370 >> v1:172.21.15.172:6789/0 conn(0x7f7a0c15e080 legacy=0x7f7a0c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a11f1c700 1 -- 172.21.15.114:0/870236370 >> v1:172.21.15.114:6789/0 conn(0x7f7a0c06c710 legacy=0x7f7a0c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a11f1c700 1 -- 172.21.15.114:0/870236370 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0ae9f0 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a11f1c700 1 --2- 172.21.15.114:0/870236370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c0af4d0 0x7f7a0c0b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009d90 tx=0x7f7a0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5a51e1d602ae03 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a03fff700 1 -- 172.21.15.114:0/870236370 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400e070 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a03fff700 1 -- 172.21.15.114:0/870236370 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0400bbd0 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.394+0000 7f7a03fff700 1 -- 172.21.15.114:0/870236370 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04012430 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 -- 172.21.15.114:0/870236370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c0af4d0 msgr2=0x7f7a0c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 --2- 172.21.15.114:0/870236370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c0af4d0 0x7f7a0c0b5050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009d90 tx=0x7f7a0400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 -- 172.21.15.114:0/870236370 shutdown_connections 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 --2- 172.21.15.114:0/870236370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c0af4d0 0x7f7a0c0b5050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 -- 172.21.15.114:0/870236370 >> 172.21.15.114:0/870236370 conn(0x7f7a0c071b90 msgr2=0x7f7a0c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 -- 172.21.15.114:0/870236370 shutdown_connections 2023-11-24T06:24:23.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.395+0000 7f7a1397f700 1 -- 172.21.15.114:0/870236370 wait complete. 2023-11-24T06:24:23.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.397+0000 7f7a1397f700 1 Processor -- start 2023-11-24T06:24:23.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.397+0000 7f7a1397f700 1 -- start start 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.397+0000 7f7a1397f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a0c06c710 0x7f7a0c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.397+0000 7f7a1397f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a0c0af4d0 0x7f7a0c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.397+0000 7f7a1397f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1397f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c160b60 con 0x7f7a0c15e080 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1397f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15ff50 con 0x7f7a0c06c710 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1397f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c160230 con 0x7f7a0c0af4d0 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37776/0 (socket says 172.21.15.114:37776) 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 -- 172.21.15.114:0/2589981711 learned_addr learned my addr 172.21.15.114:0/2589981711 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1171b700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a0c06c710 0x7f7a0c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a10f1a700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a0c0af4d0 0x7f7a0c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 -- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a0c0af4d0 msgr2=0x7f7a0c15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a0c0af4d0 0x7f7a0c15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 -- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a0c06c710 msgr2=0x7f7a0c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a0c06c710 0x7f7a0c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 -- 172.21.15.114:0/2589981711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04009a70 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a11f1c700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a04014040 tx=0x7f7a04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04012d20 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04020070 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c160510 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04012ea0 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.398+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c156320 con 0x7f7a0c15e080 2023-11-24T06:24:23.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.399+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c0753f0 con 0x7f7a0c15e080 2023-11-24T06:24:23.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.400+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a04004280 con 0x7f7a0c15e080 2023-11-24T06:24:23.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.400+0000 7f7a01ffb700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 0x7f79f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.400+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a04022070 con 0x7f7a0c15e080 2023-11-24T06:24:23.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.400+0000 7f7a1171b700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 0x7f79f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.401+0000 7f7a1171b700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 0x7f79f006d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.405+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a04056640 con 0x7f7a0c15e080 2023-11-24T06:24:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.597+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a0c067f60 con 0x7f7a0c15e080 2023-11-24T06:24:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.599+0000 7f7a01ffb700 1 -- 172.21.15.114:0/2589981711 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f7a04059c60 con 0x7f7a0c15e080 2023-11-24T06:24:23.605 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:23.606 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 msgr2=0x7f79f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 0x7f79f006d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2023-11-24T06:24:23.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 msgr2=0x7f7a0c141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a04014040 tx=0x7f7a04004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 shutdown_connections 2023-11-24T06:24:23.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79f006aed0 0x7f79f006d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a0c15e080 0x7f7a0c141b00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a0c06c710 0x7f7a0c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 --2- 172.21.15.114:0/2589981711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a0c0af4d0 0x7f7a0c15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 >> 172.21.15.114:0/2589981711 conn(0x7f7a0c071b90 msgr2=0x7f7a0c0b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 shutdown_connections 2023-11-24T06:24:23.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.601+0000 7f7a1397f700 1 -- 172.21.15.114:0/2589981711 wait complete. 2023-11-24T06:24:23.620 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:24:23.621 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:23.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.739+0000 7f12fcbc1700 1 Processor -- start 2023-11-24T06:24:23.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.739+0000 7f12fcbc1700 1 -- start start 2023-11-24T06:24:23.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.740+0000 7f12fcbc1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f12f815e080 0x7f12f80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.740+0000 7f12fcbc1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f12f806c710 0x7f12f80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.740+0000 7f12fcbc1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f80ae430 con 0x7f12f80af4d0 2023-11-24T06:24:23.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.740+0000 7f12fcbc1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f80af0b0 con 0x7f12f815e080 2023-11-24T06:24:23.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.740+0000 7f12fcbc1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f80b05a0 con 0x7f12f806c710 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f12f815e080 0x7f12f80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f5d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f12f80af4d0 0x7f12f806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53488/0 (socket says 172.21.15.114:53488) 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f5d9b700 1 -- 172.21.15.114:0/4193165288 learned_addr learned my addr 172.21.15.114:0/4193165288 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f6d9d700 1 --2- 172.21.15.114:0/4193165288 >> v2:172.21.15.172:3300/0 conn(0x7f12f806c710 0x7f12f80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 -- 172.21.15.114:0/4193165288 >> v2:172.21.15.172:3300/0 conn(0x7f12f806c710 msgr2=0x7f12f80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 --2- 172.21.15.114:0/4193165288 >> v2:172.21.15.172:3300/0 conn(0x7f12f806c710 0x7f12f80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 -- 172.21.15.114:0/4193165288 >> v1:172.21.15.114:6789/0 conn(0x7f12f80af4d0 legacy=0x7f12f806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f559a700 1 -- 172.21.15.114:0/4193165288 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352391682 0 0) 0x7f12f80ae430 con 0x7f12f80af4d0 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 -- 172.21.15.114:0/4193165288 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80ae710 con 0x7f12f815e080 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f659c700 1 --2- 172.21.15.114:0/4193165288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f815e080 0x7f12f80b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12f806e060 tx=0x7f12e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49b5965d3aa21ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f559a700 1 -- 172.21.15.114:0/4193165288 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0011830 con 0x7f12f815e080 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f559a700 1 -- 172.21.15.114:0/4193165288 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e0011990 con 0x7f12f815e080 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.741+0000 7f12f559a700 1 -- 172.21.15.114:0/4193165288 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0011d20 con 0x7f12f815e080 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 -- 172.21.15.114:0/4193165288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f815e080 msgr2=0x7f12f80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/4193165288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f815e080 0x7f12f80b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12f806e060 tx=0x7f12e0009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 -- 172.21.15.114:0/4193165288 shutdown_connections 2023-11-24T06:24:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/4193165288 >> v2:172.21.15.172:3300/0 conn(0x7f12f806c710 0x7f12f80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/4193165288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f815e080 0x7f12f80b5050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 -- 172.21.15.114:0/4193165288 >> 172.21.15.114:0/4193165288 conn(0x7f12f8071b90 msgr2=0x7f12f8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 -- 172.21.15.114:0/4193165288 shutdown_connections 2023-11-24T06:24:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.742+0000 7f12fcbc1700 1 -- 172.21.15.114:0/4193165288 wait complete. 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.743+0000 7f12fcbc1700 1 Processor -- start 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.743+0000 7f12fcbc1700 1 -- start start 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f806c710 0x7f12f8154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 0x7f12f8153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f815a220 con 0x7f12f80af4d0 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8155eb0 con 0x7f12f806c710 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12fcbc1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8156340 con 0x7f12f815e080 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37788/0 (socket says 172.21.15.114:37788) 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 -- 172.21.15.114:0/1933199300 learned_addr learned my addr 172.21.15.114:0/1933199300 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f6d9d700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 0x7f12f8153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f659c700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f806c710 0x7f12f8154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 -- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 msgr2=0x7f12f8153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 0x7f12f8153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 -- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f806c710 msgr2=0x7f12f8154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f806c710 0x7f12f8154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f5d9b700 1 -- 172.21.15.114:0/1933199300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00094a0 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.744+0000 7f12f6d9d700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 0x7f12f8153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12f5d9b700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c540 tx=0x7f12e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8004c80 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8004de0 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f81552a0 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8007310 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.745+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f8155860 con 0x7f12f80af4d0 2023-11-24T06:24:23.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.746+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f80753f0 con 0x7f12f80af4d0 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.746+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12e8007470 con 0x7f12f80af4d0 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.747+0000 7f12eeffd700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 0x7f12d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.747+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12e8089860 con 0x7f12f80af4d0 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.747+0000 7f12f659c700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 0x7f12d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.747+0000 7f12f659c700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 0x7f12d806d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12e00097c0 tx=0x7f12e001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:23.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.750+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12e80549f0 con 0x7f12f80af4d0 2023-11-24T06:24:23.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.941+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12f8067f60 con 0x7f12d806aed0 2023-11-24T06:24:23.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.948+0000 7f12eeffd700 1 -- 172.21.15.114:0/1933199300 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259383 (secure 0 0 0) 0x7f12f8067f60 con 0x7f12d806aed0 2023-11-24T06:24:23.954 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:23.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 msgr2=0x7f12d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 0x7f12d806d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12e00097c0 tx=0x7f12e001c040 comp rx=0 tx=0).stop 2023-11-24T06:24:23.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 msgr2=0x7f12f8159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:23.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c540 tx=0x7f12e800c850 comp rx=0 tx=0).stop 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 shutdown_connections 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f12d806aed0 0x7f12d806d390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12f815e080 0x7f12f8153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f806c710 0x7f12f8154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 --2- 172.21.15.114:0/1933199300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12f80af4d0 0x7f12f8159ce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 >> 172.21.15.114:0/1933199300 conn(0x7f12f8071b90 msgr2=0x7f12f815d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 shutdown_connections 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:23.951+0000 7f12fcbc1700 1 -- 172.21.15.114:0/1933199300 wait complete. 2023-11-24T06:24:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:23.966 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:24:23.708302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2703360,"data_stored":1668234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845901+0000","last_change":"2023-11-24T06:24:18.845901+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845901+0000","last_undegraded":"2023-11-24T06:24:18.845901+0000","last_fullsized":"2023-11-24T06:24:18.845901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851607+0000","last_change":"2023-11-24T06:24:18.851607+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851607+0000","last_undegraded":"2023-11-24T06:24:18.851607+0000","last_fullsized":"2023-11-24T06:24:18.851607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846763+0000","last_change":"2023-11-24T06:24:18.846763+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846763+0000","last_undegraded":"2023-11-24T06:24:18.846763+0000","last_fullsized":"2023-11-24T06:24:18.846763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850784+0000","last_change":"2023-11-24T06:24:18.850784+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850784+0000","last_undegraded":"2023-11-24T06:24:18.850784+0000","last_fullsized":"2023-11-24T06:24:18.850784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844740+0000","last_change":"2023-11-24T06:24:18.844740+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844740+0000","last_undegraded":"2023-11-24T06:24:18.844740+0000","last_fullsized":"2023-11-24T06:24:18.844740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.853282+0000","last_change":"2023-11-24T06:24:18.853282+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.853282+0000","last_undegraded":"2023-11-24T06:24:18.853282+0000","last_fullsized":"2023-11-24T06:24:18.853282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845753+0000","last_change":"2023-11-24T06:24:18.845753+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845753+0000","last_undegraded":"2023-11-24T06:24:18.845753+0000","last_fullsized":"2023-11-24T06:24:18.845753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851451+0000","last_change":"2023-11-24T06:24:18.851451+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851451+0000","last_undegraded":"2023-11-24T06:24:18.851451+0000","last_fullsized":"2023-11-24T06:24:18.851451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845084+0000","last_change":"2023-11-24T06:24:18.845084+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845084+0000","last_undegraded":"2023-11-24T06:24:18.845084+0000","last_fullsized":"2023-11-24T06:24:18.845084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849004+0000","last_change":"2023-11-24T06:24:18.849004+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849004+0000","last_undegraded":"2023-11-24T06:24:18.849004+0000","last_fullsized":"2023-11-24T06:24:18.849004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850853+0000","last_change":"2023-11-24T06:24:18.850853+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850853+0000","last_undegraded":"2023-11-24T06:24:18.850853+0000","last_fullsized":"2023-11-24T06:24:18.850853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844967+0000","last_change":"2023-11-24T06:24:18.844967+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844967+0000","last_undegraded":"2023-11-24T06:24:18.844967+0000","last_fullsized":"2023-11-24T06:24:18.844967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851767+0000","last_change":"2023-11-24T06:24:18.851767+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851767+0000","last_undegraded":"2023-11-24T06:24:18.851767+0000","last_fullsized":"2023-11-24T06:24:18.851767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846586+0000","last_change":"2023-11-24T06:24:18.846586+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846586+0000","last_undegraded":"2023-11-24T06:24:18.846586+0000","last_fullsized":"2023-11-24T06:24:18.846586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851597+0000","last_change":"2023-11-24T06:24:18.851597+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851597+0000","last_undegraded":"2023-11-24T06:24:18.851597+0000","last_fullsized":"2023-11-24T06:24:18.851597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851790+0000","last_change":"2023-11-24T06:24:18.851790+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851790+0000","last_undegraded":"2023-11-24T06:24:18.851790+0000","last_fullsized":"2023-11-24T06:24:18.851790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848814+0000","last_change":"2023-11-24T06:24:18.848814+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848814+0000","last_undegraded":"2023-11-24T06:24:18.848814+0000","last_fullsized":"2023-11-24T06:24:18.848814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850442+0000","last_change":"2023-11-24T06:24:18.850442+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850442+0000","last_undegraded":"2023-11-24T06:24:18.850442+0000","last_fullsized":"2023-11-24T06:24:18.850442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846097+0000","last_change":"2023-11-24T06:24:18.846097+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846097+0000","last_undegraded":"2023-11-24T06:24:18.846097+0000","last_fullsized":"2023-11-24T06:24:18.846097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859398+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:18.859398+0000","last_peered":"2023-11-24T06:24:18.859398+0000","last_clean":"2023-11-24T06:24:18.859398+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:18.859398+0000","last_undegraded":"2023-11-24T06:24:18.859398+0000","last_fullsized":"2023-11-24T06:24:18.859398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849020+0000","last_change":"2023-11-24T06:24:18.849020+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849020+0000","last_undegraded":"2023-11-24T06:24:18.849020+0000","last_fullsized":"2023-11-24T06:24:18.849020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859724+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:18.859724+0000","last_peered":"2023-11-24T06:24:18.859724+0000","last_clean":"2023-11-24T06:24:18.859724+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:18.859724+0000","last_undegraded":"2023-11-24T06:24:18.859724+0000","last_fullsized":"2023-11-24T06:24:18.859724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847546+0000","last_change":"2023-11-24T06:24:18.847546+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847546+0000","last_undegraded":"2023-11-24T06:24:18.847546+0000","last_fullsized":"2023-11-24T06:24:18.847546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862355+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:18.862355+0000","last_peered":"2023-11-24T06:24:18.862355+0000","last_clean":"2023-11-24T06:24:18.862355+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:18.862355+0000","last_undegraded":"2023-11-24T06:24:18.862355+0000","last_fullsized":"2023-11-24T06:24:18.862355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.856218+0000","last_change":"2023-11-24T06:24:18.856218+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.856218+0000","last_undegraded":"2023-11-24T06:24:18.856218+0000","last_fullsized":"2023-11-24T06:24:18.856218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864140+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:18.864140+0000","last_peered":"2023-11-24T06:24:18.864140+0000","last_clean":"2023-11-24T06:24:18.864140+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:18.864140+0000","last_undegraded":"2023-11-24T06:24:18.864140+0000","last_fullsized":"2023-11-24T06:24:18.864140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850116+0000","last_change":"2023-11-24T06:24:18.850116+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850116+0000","last_undegraded":"2023-11-24T06:24:18.850116+0000","last_fullsized":"2023-11-24T06:24:18.850116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865327+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:18.865327+0000","last_peered":"2023-11-24T06:24:18.865327+0000","last_clean":"2023-11-24T06:24:18.865327+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:18.865327+0000","last_undegraded":"2023-11-24T06:24:18.865327+0000","last_fullsized":"2023-11-24T06:24:18.865327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843310+0000","last_change":"2023-11-24T06:24:18.843310+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843310+0000","last_undegraded":"2023-11-24T06:24:18.843310+0000","last_fullsized":"2023-11-24T06:24:18.843310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859116+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:18.859116+0000","last_peered":"2023-11-24T06:24:18.859116+0000","last_clean":"2023-11-24T06:24:18.859116+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:18.859116+0000","last_undegraded":"2023-11-24T06:24:18.859116+0000","last_fullsized":"2023-11-24T06:24:18.859116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845473+0000","last_change":"2023-11-24T06:24:18.845473+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845473+0000","last_undegraded":"2023-11-24T06:24:18.845473+0000","last_fullsized":"2023-11-24T06:24:18.845473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859717+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:18.859717+0000","last_peered":"2023-11-24T06:24:18.859717+0000","last_clean":"2023-11-24T06:24:18.859717+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:18.859717+0000","last_undegraded":"2023-11-24T06:24:18.859717+0000","last_fullsized":"2023-11-24T06:24:18.859717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845987+0000","last_change":"2023-11-24T06:24:18.845987+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845987+0000","last_undegraded":"2023-11-24T06:24:18.845987+0000","last_fullsized":"2023-11-24T06:24:18.845987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863686+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:18.863686+0000","last_peered":"2023-11-24T06:24:18.863686+0000","last_clean":"2023-11-24T06:24:18.863686+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:18.863686+0000","last_undegraded":"2023-11-24T06:24:18.863686+0000","last_fullsized":"2023-11-24T06:24:18.863686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850409+0000","last_change":"2023-11-24T06:24:18.850409+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850409+0000","last_undegraded":"2023-11-24T06:24:18.850409+0000","last_fullsized":"2023-11-24T06:24:18.850409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864305+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:18.864305+0000","last_peered":"2023-11-24T06:24:18.864305+0000","last_clean":"2023-11-24T06:24:18.864305+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:18.864305+0000","last_undegraded":"2023-11-24T06:24:18.864305+0000","last_fullsized":"2023-11-24T06:24:18.864305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850760+0000","last_change":"2023-11-24T06:24:18.850760+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850760+0000","last_undegraded":"2023-11-24T06:24:18.850760+0000","last_fullsized":"2023-11-24T06:24:18.850760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863658+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:18.863658+0000","last_peered":"2023-11-24T06:24:18.863658+0000","last_clean":"2023-11-24T06:24:18.863658+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:18.863658+0000","last_undegraded":"2023-11-24T06:24:18.863658+0000","last_fullsized":"2023-11-24T06:24:18.863658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847096+0000","last_change":"2023-11-24T06:24:18.847096+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847096+0000","last_undegraded":"2023-11-24T06:24:18.847096+0000","last_fullsized":"2023-11-24T06:24:18.847096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.866421+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:18.866421+0000","last_peered":"2023-11-24T06:24:18.866421+0000","last_clean":"2023-11-24T06:24:18.866421+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:18.866421+0000","last_undegraded":"2023-11-24T06:24:18.866421+0000","last_fullsized":"2023-11-24T06:24:18.866421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843113+0000","last_change":"2023-11-24T06:24:18.843113+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843113+0000","last_undegraded":"2023-11-24T06:24:18.843113+0000","last_fullsized":"2023-11-24T06:24:18.843113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863776+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:18.863776+0000","last_peered":"2023-11-24T06:24:18.863776+0000","last_clean":"2023-11-24T06:24:18.863776+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:18.863776+0000","last_undegraded":"2023-11-24T06:24:18.863776+0000","last_fullsized":"2023-11-24T06:24:18.863776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845117+0000","last_change":"2023-11-24T06:24:18.845117+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845117+0000","last_undegraded":"2023-11-24T06:24:18.845117+0000","last_fullsized":"2023-11-24T06:24:18.845117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.856003+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:18.856003+0000","last_peered":"2023-11-24T06:24:18.856003+0000","last_clean":"2023-11-24T06:24:18.856003+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:18.856003+0000","last_undegraded":"2023-11-24T06:24:18.856003+0000","last_fullsized":"2023-11-24T06:24:18.856003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845031+0000","last_change":"2023-11-24T06:24:18.845031+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845031+0000","last_undegraded":"2023-11-24T06:24:18.845031+0000","last_fullsized":"2023-11-24T06:24:18.845031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863773+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:18.863773+0000","last_peered":"2023-11-24T06:24:18.863773+0000","last_clean":"2023-11-24T06:24:18.863773+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:18.863773+0000","last_undegraded":"2023-11-24T06:24:18.863773+0000","last_fullsized":"2023-11-24T06:24:18.863773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843918+0000","last_change":"2023-11-24T06:24:18.843918+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843918+0000","last_undegraded":"2023-11-24T06:24:18.843918+0000","last_fullsized":"2023-11-24T06:24:18.843918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859552+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:18.859552+0000","last_peered":"2023-11-24T06:24:18.859552+0000","last_clean":"2023-11-24T06:24:18.859552+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:18.859552+0000","last_undegraded":"2023-11-24T06:24:18.859552+0000","last_fullsized":"2023-11-24T06:24:18.859552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849274+0000","last_change":"2023-11-24T06:24:18.849274+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849274+0000","last_undegraded":"2023-11-24T06:24:18.849274+0000","last_fullsized":"2023-11-24T06:24:18.849274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865485+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:18.865485+0000","last_peered":"2023-11-24T06:24:18.865485+0000","last_clean":"2023-11-24T06:24:18.865485+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:18.865485+0000","last_undegraded":"2023-11-24T06:24:18.865485+0000","last_fullsized":"2023-11-24T06:24:18.865485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844048+0000","last_change":"2023-11-24T06:24:18.844048+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844048+0000","last_undegraded":"2023-11-24T06:24:18.844048+0000","last_fullsized":"2023-11-24T06:24:18.844048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.858819+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:18.858819+0000","last_peered":"2023-11-24T06:24:18.858819+0000","last_clean":"2023-11-24T06:24:18.858819+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:18.858819+0000","last_undegraded":"2023-11-24T06:24:18.858819+0000","last_fullsized":"2023-11-24T06:24:18.858819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844498+0000","last_change":"2023-11-24T06:24:18.844498+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844498+0000","last_undegraded":"2023-11-24T06:24:18.844498+0000","last_fullsized":"2023-11-24T06:24:18.844498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864328+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:18.864328+0000","last_peered":"2023-11-24T06:24:18.864328+0000","last_clean":"2023-11-24T06:24:18.864328+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:18.864328+0000","last_undegraded":"2023-11-24T06:24:18.864328+0000","last_fullsized":"2023-11-24T06:24:18.864328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851522+0000","last_change":"2023-11-24T06:24:18.851522+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851522+0000","last_undegraded":"2023-11-24T06:24:18.851522+0000","last_fullsized":"2023-11-24T06:24:18.851522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864303+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:18.864303+0000","last_peered":"2023-11-24T06:24:18.864303+0000","last_clean":"2023-11-24T06:24:18.864303+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:18.864303+0000","last_undegraded":"2023-11-24T06:24:18.864303+0000","last_fullsized":"2023-11-24T06:24:18.864303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843387+0000","last_change":"2023-11-24T06:24:18.843387+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843387+0000","last_undegraded":"2023-11-24T06:24:18.843387+0000","last_fullsized":"2023-11-24T06:24:18.843387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848580+0000","last_change":"2023-11-24T06:24:18.848580+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848580+0000","last_undegraded":"2023-11-24T06:24:18.848580+0000","last_fullsized":"2023-11-24T06:24:18.848580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864891+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:18.864891+0000","last_peered":"2023-11-24T06:24:18.864891+0000","last_clean":"2023-11-24T06:24:18.864891+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:18.864891+0000","last_undegraded":"2023-11-24T06:24:18.864891+0000","last_fullsized":"2023-11-24T06:24:18.864891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845534+0000","last_change":"2023-11-24T06:24:18.845534+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845534+0000","last_undegraded":"2023-11-24T06:24:18.845534+0000","last_fullsized":"2023-11-24T06:24:18.845534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864162+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:18.864162+0000","last_peered":"2023-11-24T06:24:18.864162+0000","last_clean":"2023-11-24T06:24:18.864162+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:18.864162+0000","last_undegraded":"2023-11-24T06:24:18.864162+0000","last_fullsized":"2023-11-24T06:24:18.864162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:24:17.839121+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:17.839121+0000","last_peered":"2023-11-24T06:24:17.839121+0000","last_clean":"2023-11-24T06:24:17.839121+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:17.839121+0000","last_undegraded":"2023-11-24T06:24:17.839121+0000","last_fullsized":"2023-11-24T06:24:17.839121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844436+0000","last_change":"2023-11-24T06:24:18.844436+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844436+0000","last_undegraded":"2023-11-24T06:24:18.844436+0000","last_fullsized":"2023-11-24T06:24:18.844436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861746+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:18.861746+0000","last_peered":"2023-11-24T06:24:18.861746+0000","last_clean":"2023-11-24T06:24:18.861746+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:18.861746+0000","last_undegraded":"2023-11-24T06:24:18.861746+0000","last_fullsized":"2023-11-24T06:24:18.861746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862011+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:18.862011+0000","last_peered":"2023-11-24T06:24:18.862011+0000","last_clean":"2023-11-24T06:24:18.862011+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:18.862011+0000","last_undegraded":"2023-11-24T06:24:18.862011+0000","last_fullsized":"2023-11-24T06:24:18.862011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850677+0000","last_change":"2023-11-24T06:24:18.850677+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850677+0000","last_undegraded":"2023-11-24T06:24:18.850677+0000","last_fullsized":"2023-11-24T06:24:18.850677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861727+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:18.861727+0000","last_peered":"2023-11-24T06:24:18.861727+0000","last_clean":"2023-11-24T06:24:18.861727+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:18.861727+0000","last_undegraded":"2023-11-24T06:24:18.861727+0000","last_fullsized":"2023-11-24T06:24:18.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849533+0000","last_change":"2023-11-24T06:24:18.849533+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849533+0000","last_undegraded":"2023-11-24T06:24:18.849533+0000","last_fullsized":"2023-11-24T06:24:18.849533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864318+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:18.864318+0000","last_peered":"2023-11-24T06:24:18.864318+0000","last_clean":"2023-11-24T06:24:18.864318+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:18.864318+0000","last_undegraded":"2023-11-24T06:24:18.864318+0000","last_fullsized":"2023-11-24T06:24:18.864318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844472+0000","last_change":"2023-11-24T06:24:18.844472+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844472+0000","last_undegraded":"2023-11-24T06:24:18.844472+0000","last_fullsized":"2023-11-24T06:24:18.844472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862045+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:18.862045+0000","last_peered":"2023-11-24T06:24:18.862045+0000","last_clean":"2023-11-24T06:24:18.862045+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:18.862045+0000","last_undegraded":"2023-11-24T06:24:18.862045+0000","last_fullsized":"2023-11-24T06:24:18.862045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843078+0000","last_change":"2023-11-24T06:24:18.843078+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843078+0000","last_undegraded":"2023-11-24T06:24:18.843078+0000","last_fullsized":"2023-11-24T06:24:18.843078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863798+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:18.863798+0000","last_peered":"2023-11-24T06:24:18.863798+0000","last_clean":"2023-11-24T06:24:18.863798+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:18.863798+0000","last_undegraded":"2023-11-24T06:24:18.863798+0000","last_fullsized":"2023-11-24T06:24:18.863798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845239+0000","last_change":"2023-11-24T06:24:18.845239+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845239+0000","last_undegraded":"2023-11-24T06:24:18.845239+0000","last_fullsized":"2023-11-24T06:24:18.845239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864361+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:18.864361+0000","last_peered":"2023-11-24T06:24:18.864361+0000","last_clean":"2023-11-24T06:24:18.864361+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:18.864361+0000","last_undegraded":"2023-11-24T06:24:18.864361+0000","last_fullsized":"2023-11-24T06:24:18.864361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845746+0000","last_change":"2023-11-24T06:24:18.845746+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845746+0000","last_undegraded":"2023-11-24T06:24:18.845746+0000","last_fullsized":"2023-11-24T06:24:18.845746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861865+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:18.861865+0000","last_peered":"2023-11-24T06:24:18.861865+0000","last_clean":"2023-11-24T06:24:18.861865+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:18.861865+0000","last_undegraded":"2023-11-24T06:24:18.861865+0000","last_fullsized":"2023-11-24T06:24:18.861865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850115+0000","last_change":"2023-11-24T06:24:18.850115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850115+0000","last_undegraded":"2023-11-24T06:24:18.850115+0000","last_fullsized":"2023-11-24T06:24:18.850115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865516+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:18.865516+0000","last_peered":"2023-11-24T06:24:18.865516+0000","last_clean":"2023-11-24T06:24:18.865516+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:18.865516+0000","last_undegraded":"2023-11-24T06:24:18.865516+0000","last_fullsized":"2023-11-24T06:24:18.865516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844391+0000","last_change":"2023-11-24T06:24:18.844391+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844391+0000","last_undegraded":"2023-11-24T06:24:18.844391+0000","last_fullsized":"2023-11-24T06:24:18.844391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865112+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:18.865112+0000","last_peered":"2023-11-24T06:24:18.865112+0000","last_clean":"2023-11-24T06:24:18.865112+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:18.865112+0000","last_undegraded":"2023-11-24T06:24:18.865112+0000","last_fullsized":"2023-11-24T06:24:18.865112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854299+0000","last_change":"2023-11-24T06:24:18.854299+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854299+0000","last_undegraded":"2023-11-24T06:24:18.854299+0000","last_fullsized":"2023-11-24T06:24:18.854299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862641+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:18.862641+0000","last_peered":"2023-11-24T06:24:18.862641+0000","last_clean":"2023-11-24T06:24:18.862641+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:18.862641+0000","last_undegraded":"2023-11-24T06:24:18.862641+0000","last_fullsized":"2023-11-24T06:24:18.862641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843946+0000","last_change":"2023-11-24T06:24:18.843946+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843946+0000","last_undegraded":"2023-11-24T06:24:18.843946+0000","last_fullsized":"2023-11-24T06:24:18.843946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854564+0000","last_change":"2023-11-24T06:24:18.854564+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854564+0000","last_undegraded":"2023-11-24T06:24:18.854564+0000","last_fullsized":"2023-11-24T06:24:18.854564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850064+0000","last_change":"2023-11-24T06:24:18.850064+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850064+0000","last_undegraded":"2023-11-24T06:24:18.850064+0000","last_fullsized":"2023-11-24T06:24:18.850064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850695+0000","last_change":"2023-11-24T06:24:18.850695+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850695+0000","last_undegraded":"2023-11-24T06:24:18.850695+0000","last_fullsized":"2023-11-24T06:24:18.850695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845998+0000","last_change":"2023-11-24T06:24:18.845998+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845998+0000","last_undegraded":"2023-11-24T06:24:18.845998+0000","last_fullsized":"2023-11-24T06:24:18.845998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849115+0000","last_change":"2023-11-24T06:24:18.849115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849115+0000","last_undegraded":"2023-11-24T06:24:18.849115+0000","last_fullsized":"2023-11-24T06:24:18.849115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851388+0000","last_change":"2023-11-24T06:24:18.851388+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851388+0000","last_undegraded":"2023-11-24T06:24:18.851388+0000","last_fullsized":"2023-11-24T06:24:18.851388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851520+0000","last_change":"2023-11-24T06:24:18.851520+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851520+0000","last_undegraded":"2023-11-24T06:24:18.851520+0000","last_fullsized":"2023-11-24T06:24:18.851520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849982+0000","last_change":"2023-11-24T06:24:18.849982+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849982+0000","last_undegraded":"2023-11-24T06:24:18.849982+0000","last_fullsized":"2023-11-24T06:24:18.849982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847930+0000","last_change":"2023-11-24T06:24:18.847930+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847930+0000","last_undegraded":"2023-11-24T06:24:18.847930+0000","last_fullsized":"2023-11-24T06:24:18.847930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850083+0000","last_change":"2023-11-24T06:24:18.850083+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850083+0000","last_undegraded":"2023-11-24T06:24:18.850083+0000","last_fullsized":"2023-11-24T06:24:18.850083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851624+0000","last_change":"2023-11-24T06:24:18.851624+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851624+0000","last_undegraded":"2023-11-24T06:24:18.851624+0000","last_fullsized":"2023-11-24T06:24:18.851624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846419+0000","last_change":"2023-11-24T06:24:18.846419+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846419+0000","last_undegraded":"2023-11-24T06:24:18.846419+0000","last_fullsized":"2023-11-24T06:24:18.846419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849459+0000","last_change":"2023-11-24T06:24:18.849459+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849459+0000","last_undegraded":"2023-11-24T06:24:18.849459+0000","last_fullsized":"2023-11-24T06:24:18.849459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":646599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":652874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:23.972 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:24.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.124+0000 7f5561cd8700 1 Processor -- start 2023-11-24T06:24:24.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.124+0000 7f5561cd8700 1 -- start start 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5561cd8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f555c0b3370 0x7f555c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5561cd8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f555c0d1c70 0x7f555c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5561cd8700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c0b22d0 con 0x7f555c15bdd0 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5561cd8700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0b2f50 con 0x7f555c0d1c70 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5561cd8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0d2640 con 0x7f555c0b3370 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f555b7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f555c15bdd0 0x7f555c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53502/0 (socket says 172.21.15.114:53502) 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f555b7fe700 1 -- 172.21.15.114:0/228297553 learned_addr learned my addr 172.21.15.114:0/228297553 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f555bfff700 1 --2- 172.21.15.114:0/228297553 >> v2:172.21.15.114:3300/0 conn(0x7f555c0d1c70 0x7f555c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f555affd700 1 -- 172.21.15.114:0/228297553 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358863945 0 0) 0x7f555c0b22d0 con 0x7f555c15bdd0 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555affd700 1 -- 172.21.15.114:0/228297553 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55480035b0 con 0x7f555c15bdd0 2023-11-24T06:24:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.125+0000 7f5552ffd700 1 --2- 172.21.15.114:0/228297553 >> v2:172.21.15.172:3300/0 conn(0x7f555c0b3370 0x7f555c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555bfff700 1 -- 172.21.15.114:0/228297553 >> v2:172.21.15.172:3300/0 conn(0x7f555c0b3370 msgr2=0x7f555c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555bfff700 1 --2- 172.21.15.114:0/228297553 >> v2:172.21.15.172:3300/0 conn(0x7f555c0b3370 0x7f555c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555bfff700 1 -- 172.21.15.114:0/228297553 >> v1:172.21.15.114:6789/0 conn(0x7f555c15bdd0 legacy=0x7f555c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555bfff700 1 -- 172.21.15.114:0/228297553 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0b25b0 con 0x7f555c0d1c70 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555bfff700 1 --2- 172.21.15.114:0/228297553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c0d1c70 0x7f555c0d2100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f555c15f910 tx=0x7f555400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b710cb08bc86f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555affd700 1 -- 172.21.15.114:0/228297553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555400e070 con 0x7f555c0d1c70 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555affd700 1 -- 172.21.15.114:0/228297553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555400bb20 con 0x7f555c0d1c70 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f555affd700 1 -- 172.21.15.114:0/228297553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554012430 con 0x7f555c0d1c70 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f5561cd8700 1 -- 172.21.15.114:0/228297553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c0d1c70 msgr2=0x7f555c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f5561cd8700 1 --2- 172.21.15.114:0/228297553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c0d1c70 0x7f555c0d2100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f555c15f910 tx=0x7f555400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f5561cd8700 1 -- 172.21.15.114:0/228297553 shutdown_connections 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f5561cd8700 1 --2- 172.21.15.114:0/228297553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c0d1c70 0x7f555c0d2100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.126+0000 7f5561cd8700 1 --2- 172.21.15.114:0/228297553 >> v2:172.21.15.172:3300/0 conn(0x7f555c0b3370 0x7f555c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.127+0000 7f5561cd8700 1 -- 172.21.15.114:0/228297553 >> 172.21.15.114:0/228297553 conn(0x7f555c0731a0 msgr2=0x7f555c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:24.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.127+0000 7f5561cd8700 1 -- 172.21.15.114:0/228297553 shutdown_connections 2023-11-24T06:24:24.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.127+0000 7f5561cd8700 1 -- 172.21.15.114:0/228297553 wait complete. 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.128+0000 7f5561cd8700 1 Processor -- start 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.128+0000 7f5561cd8700 1 -- start start 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f555c0b3370 0x7f555c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f555c0d1c70 0x7f555c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c06bfe0 con 0x7f555c15bdd0 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c06b360 con 0x7f555c0d1c70 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c06b640 con 0x7f555c0b3370 2023-11-24T06:24:24.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37804/0 (socket says 172.21.15.114:37804) 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 -- 172.21.15.114:0/1196729814 learned_addr learned my addr 172.21.15.114:0/1196729814 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5552ffd700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f555c0d1c70 0x7f555c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555b7fe700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f555c0b3370 0x7f555c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 -- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f555c0b3370 msgr2=0x7f555c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f555c0b3370 0x7f555c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 -- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f555c0d1c70 msgr2=0x7f555c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f555c0d1c70 0x7f555c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 -- 172.21.15.114:0/1196729814 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554009a70 con 0x7f555c15bdd0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f555bfff700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5554014040 tx=0x7f5554004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554012430 con 0x7f555c15bdd0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554020070 con 0x7f555c15bdd0 2023-11-24T06:24:24.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c06b920 con 0x7f555c15bdd0 2023-11-24T06:24:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554004280 con 0x7f555c15bdd0 2023-11-24T06:24:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.129+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c06aff0 con 0x7f555c15bdd0 2023-11-24T06:24:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.130+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c06c120 con 0x7f555c15bdd0 2023-11-24T06:24:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.131+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55540043e0 con 0x7f555c15bdd0 2023-11-24T06:24:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.131+0000 7f5558ff9700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 0x7f553c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.131+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5554022070 con 0x7f555c15bdd0 2023-11-24T06:24:24.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.131+0000 7f555b7fe700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 0x7f553c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.131+0000 7f555b7fe700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 0x7f553c06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f554c0011e0 tx=0x7f554c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.135+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f555c06c120 con 0x7f555c15bdd0 2023-11-24T06:24:24.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.320+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f555c0b2b70 con 0x7f553c06aed0 2023-11-24T06:24:24.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.327+0000 7f5558ff9700 1 -- 172.21.15.114:0/1196729814 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259383 (secure 0 0 0) 0x7f554c001460 con 0x7f553c06aed0 2023-11-24T06:24:24.336 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:24.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 msgr2=0x7f553c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 0x7f553c06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f554c0011e0 tx=0x7f554c009040 comp rx=0 tx=0).stop 2023-11-24T06:24:24.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 msgr2=0x7f555c15a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5554014040 tx=0x7f5554004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 shutdown_connections 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f553c06aed0 0x7f553c06d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f555c15bdd0 0x7f555c15a430 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f555c0b3370 0x7f555c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 --2- 172.21.15.114:0/1196729814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f555c0d1c70 0x7f555c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 >> 172.21.15.114:0/1196729814 conn(0x7f555c0731a0 msgr2=0x7f555c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:24.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 shutdown_connections 2023-11-24T06:24:24.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.330+0000 7f5561cd8700 1 -- 172.21.15.114:0/1196729814 wait complete. 2023-11-24T06:24:24.342 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:24.346 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:24:23.708302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2703360,"data_stored":1668234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845901+0000","last_change":"2023-11-24T06:24:18.845901+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845901+0000","last_undegraded":"2023-11-24T06:24:18.845901+0000","last_fullsized":"2023-11-24T06:24:18.845901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851607+0000","last_change":"2023-11-24T06:24:18.851607+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851607+0000","last_undegraded":"2023-11-24T06:24:18.851607+0000","last_fullsized":"2023-11-24T06:24:18.851607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846763+0000","last_change":"2023-11-24T06:24:18.846763+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846763+0000","last_undegraded":"2023-11-24T06:24:18.846763+0000","last_fullsized":"2023-11-24T06:24:18.846763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850784+0000","last_change":"2023-11-24T06:24:18.850784+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850784+0000","last_undegraded":"2023-11-24T06:24:18.850784+0000","last_fullsized":"2023-11-24T06:24:18.850784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844740+0000","last_change":"2023-11-24T06:24:18.844740+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844740+0000","last_undegraded":"2023-11-24T06:24:18.844740+0000","last_fullsized":"2023-11-24T06:24:18.844740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.853282+0000","last_change":"2023-11-24T06:24:18.853282+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.853282+0000","last_undegraded":"2023-11-24T06:24:18.853282+0000","last_fullsized":"2023-11-24T06:24:18.853282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845753+0000","last_change":"2023-11-24T06:24:18.845753+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845753+0000","last_undegraded":"2023-11-24T06:24:18.845753+0000","last_fullsized":"2023-11-24T06:24:18.845753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851451+0000","last_change":"2023-11-24T06:24:18.851451+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851451+0000","last_undegraded":"2023-11-24T06:24:18.851451+0000","last_fullsized":"2023-11-24T06:24:18.851451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845084+0000","last_change":"2023-11-24T06:24:18.845084+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845084+0000","last_undegraded":"2023-11-24T06:24:18.845084+0000","last_fullsized":"2023-11-24T06:24:18.845084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849004+0000","last_change":"2023-11-24T06:24:18.849004+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849004+0000","last_undegraded":"2023-11-24T06:24:18.849004+0000","last_fullsized":"2023-11-24T06:24:18.849004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850853+0000","last_change":"2023-11-24T06:24:18.850853+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850853+0000","last_undegraded":"2023-11-24T06:24:18.850853+0000","last_fullsized":"2023-11-24T06:24:18.850853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844967+0000","last_change":"2023-11-24T06:24:18.844967+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844967+0000","last_undegraded":"2023-11-24T06:24:18.844967+0000","last_fullsized":"2023-11-24T06:24:18.844967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851767+0000","last_change":"2023-11-24T06:24:18.851767+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851767+0000","last_undegraded":"2023-11-24T06:24:18.851767+0000","last_fullsized":"2023-11-24T06:24:18.851767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846586+0000","last_change":"2023-11-24T06:24:18.846586+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846586+0000","last_undegraded":"2023-11-24T06:24:18.846586+0000","last_fullsized":"2023-11-24T06:24:18.846586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851597+0000","last_change":"2023-11-24T06:24:18.851597+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851597+0000","last_undegraded":"2023-11-24T06:24:18.851597+0000","last_fullsized":"2023-11-24T06:24:18.851597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851790+0000","last_change":"2023-11-24T06:24:18.851790+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851790+0000","last_undegraded":"2023-11-24T06:24:18.851790+0000","last_fullsized":"2023-11-24T06:24:18.851790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848814+0000","last_change":"2023-11-24T06:24:18.848814+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848814+0000","last_undegraded":"2023-11-24T06:24:18.848814+0000","last_fullsized":"2023-11-24T06:24:18.848814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850442+0000","last_change":"2023-11-24T06:24:18.850442+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850442+0000","last_undegraded":"2023-11-24T06:24:18.850442+0000","last_fullsized":"2023-11-24T06:24:18.850442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846097+0000","last_change":"2023-11-24T06:24:18.846097+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846097+0000","last_undegraded":"2023-11-24T06:24:18.846097+0000","last_fullsized":"2023-11-24T06:24:18.846097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859398+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:18.859398+0000","last_peered":"2023-11-24T06:24:18.859398+0000","last_clean":"2023-11-24T06:24:18.859398+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:18.859398+0000","last_undegraded":"2023-11-24T06:24:18.859398+0000","last_fullsized":"2023-11-24T06:24:18.859398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849020+0000","last_change":"2023-11-24T06:24:18.849020+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849020+0000","last_undegraded":"2023-11-24T06:24:18.849020+0000","last_fullsized":"2023-11-24T06:24:18.849020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859724+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:18.859724+0000","last_peered":"2023-11-24T06:24:18.859724+0000","last_clean":"2023-11-24T06:24:18.859724+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:18.859724+0000","last_undegraded":"2023-11-24T06:24:18.859724+0000","last_fullsized":"2023-11-24T06:24:18.859724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847546+0000","last_change":"2023-11-24T06:24:18.847546+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847546+0000","last_undegraded":"2023-11-24T06:24:18.847546+0000","last_fullsized":"2023-11-24T06:24:18.847546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862355+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:18.862355+0000","last_peered":"2023-11-24T06:24:18.862355+0000","last_clean":"2023-11-24T06:24:18.862355+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:18.862355+0000","last_undegraded":"2023-11-24T06:24:18.862355+0000","last_fullsized":"2023-11-24T06:24:18.862355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.856218+0000","last_change":"2023-11-24T06:24:18.856218+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.856218+0000","last_undegraded":"2023-11-24T06:24:18.856218+0000","last_fullsized":"2023-11-24T06:24:18.856218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864140+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:18.864140+0000","last_peered":"2023-11-24T06:24:18.864140+0000","last_clean":"2023-11-24T06:24:18.864140+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:18.864140+0000","last_undegraded":"2023-11-24T06:24:18.864140+0000","last_fullsized":"2023-11-24T06:24:18.864140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850116+0000","last_change":"2023-11-24T06:24:18.850116+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850116+0000","last_undegraded":"2023-11-24T06:24:18.850116+0000","last_fullsized":"2023-11-24T06:24:18.850116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865327+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:18.865327+0000","last_peered":"2023-11-24T06:24:18.865327+0000","last_clean":"2023-11-24T06:24:18.865327+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:18.865327+0000","last_undegraded":"2023-11-24T06:24:18.865327+0000","last_fullsized":"2023-11-24T06:24:18.865327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843310+0000","last_change":"2023-11-24T06:24:18.843310+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843310+0000","last_undegraded":"2023-11-24T06:24:18.843310+0000","last_fullsized":"2023-11-24T06:24:18.843310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859116+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:18.859116+0000","last_peered":"2023-11-24T06:24:18.859116+0000","last_clean":"2023-11-24T06:24:18.859116+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:18.859116+0000","last_undegraded":"2023-11-24T06:24:18.859116+0000","last_fullsized":"2023-11-24T06:24:18.859116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845473+0000","last_change":"2023-11-24T06:24:18.845473+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845473+0000","last_undegraded":"2023-11-24T06:24:18.845473+0000","last_fullsized":"2023-11-24T06:24:18.845473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859717+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:18.859717+0000","last_peered":"2023-11-24T06:24:18.859717+0000","last_clean":"2023-11-24T06:24:18.859717+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:18.859717+0000","last_undegraded":"2023-11-24T06:24:18.859717+0000","last_fullsized":"2023-11-24T06:24:18.859717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845987+0000","last_change":"2023-11-24T06:24:18.845987+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845987+0000","last_undegraded":"2023-11-24T06:24:18.845987+0000","last_fullsized":"2023-11-24T06:24:18.845987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863686+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:18.863686+0000","last_peered":"2023-11-24T06:24:18.863686+0000","last_clean":"2023-11-24T06:24:18.863686+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:18.863686+0000","last_undegraded":"2023-11-24T06:24:18.863686+0000","last_fullsized":"2023-11-24T06:24:18.863686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850409+0000","last_change":"2023-11-24T06:24:18.850409+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850409+0000","last_undegraded":"2023-11-24T06:24:18.850409+0000","last_fullsized":"2023-11-24T06:24:18.850409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864305+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:18.864305+0000","last_peered":"2023-11-24T06:24:18.864305+0000","last_clean":"2023-11-24T06:24:18.864305+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:18.864305+0000","last_undegraded":"2023-11-24T06:24:18.864305+0000","last_fullsized":"2023-11-24T06:24:18.864305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850760+0000","last_change":"2023-11-24T06:24:18.850760+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850760+0000","last_undegraded":"2023-11-24T06:24:18.850760+0000","last_fullsized":"2023-11-24T06:24:18.850760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863658+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:18.863658+0000","last_peered":"2023-11-24T06:24:18.863658+0000","last_clean":"2023-11-24T06:24:18.863658+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:18.863658+0000","last_undegraded":"2023-11-24T06:24:18.863658+0000","last_fullsized":"2023-11-24T06:24:18.863658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847096+0000","last_change":"2023-11-24T06:24:18.847096+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847096+0000","last_undegraded":"2023-11-24T06:24:18.847096+0000","last_fullsized":"2023-11-24T06:24:18.847096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.866421+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:18.866421+0000","last_peered":"2023-11-24T06:24:18.866421+0000","last_clean":"2023-11-24T06:24:18.866421+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:18.866421+0000","last_undegraded":"2023-11-24T06:24:18.866421+0000","last_fullsized":"2023-11-24T06:24:18.866421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843113+0000","last_change":"2023-11-24T06:24:18.843113+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843113+0000","last_undegraded":"2023-11-24T06:24:18.843113+0000","last_fullsized":"2023-11-24T06:24:18.843113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863776+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:18.863776+0000","last_peered":"2023-11-24T06:24:18.863776+0000","last_clean":"2023-11-24T06:24:18.863776+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:18.863776+0000","last_undegraded":"2023-11-24T06:24:18.863776+0000","last_fullsized":"2023-11-24T06:24:18.863776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845117+0000","last_change":"2023-11-24T06:24:18.845117+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845117+0000","last_undegraded":"2023-11-24T06:24:18.845117+0000","last_fullsized":"2023-11-24T06:24:18.845117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.856003+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:18.856003+0000","last_peered":"2023-11-24T06:24:18.856003+0000","last_clean":"2023-11-24T06:24:18.856003+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:18.856003+0000","last_undegraded":"2023-11-24T06:24:18.856003+0000","last_fullsized":"2023-11-24T06:24:18.856003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845031+0000","last_change":"2023-11-24T06:24:18.845031+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845031+0000","last_undegraded":"2023-11-24T06:24:18.845031+0000","last_fullsized":"2023-11-24T06:24:18.845031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863773+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:18.863773+0000","last_peered":"2023-11-24T06:24:18.863773+0000","last_clean":"2023-11-24T06:24:18.863773+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:18.863773+0000","last_undegraded":"2023-11-24T06:24:18.863773+0000","last_fullsized":"2023-11-24T06:24:18.863773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843918+0000","last_change":"2023-11-24T06:24:18.843918+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843918+0000","last_undegraded":"2023-11-24T06:24:18.843918+0000","last_fullsized":"2023-11-24T06:24:18.843918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859552+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:18.859552+0000","last_peered":"2023-11-24T06:24:18.859552+0000","last_clean":"2023-11-24T06:24:18.859552+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:18.859552+0000","last_undegraded":"2023-11-24T06:24:18.859552+0000","last_fullsized":"2023-11-24T06:24:18.859552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849274+0000","last_change":"2023-11-24T06:24:18.849274+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849274+0000","last_undegraded":"2023-11-24T06:24:18.849274+0000","last_fullsized":"2023-11-24T06:24:18.849274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865485+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:18.865485+0000","last_peered":"2023-11-24T06:24:18.865485+0000","last_clean":"2023-11-24T06:24:18.865485+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:18.865485+0000","last_undegraded":"2023-11-24T06:24:18.865485+0000","last_fullsized":"2023-11-24T06:24:18.865485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844048+0000","last_change":"2023-11-24T06:24:18.844048+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844048+0000","last_undegraded":"2023-11-24T06:24:18.844048+0000","last_fullsized":"2023-11-24T06:24:18.844048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.858819+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:18.858819+0000","last_peered":"2023-11-24T06:24:18.858819+0000","last_clean":"2023-11-24T06:24:18.858819+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:18.858819+0000","last_undegraded":"2023-11-24T06:24:18.858819+0000","last_fullsized":"2023-11-24T06:24:18.858819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844498+0000","last_change":"2023-11-24T06:24:18.844498+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844498+0000","last_undegraded":"2023-11-24T06:24:18.844498+0000","last_fullsized":"2023-11-24T06:24:18.844498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864328+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:18.864328+0000","last_peered":"2023-11-24T06:24:18.864328+0000","last_clean":"2023-11-24T06:24:18.864328+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:18.864328+0000","last_undegraded":"2023-11-24T06:24:18.864328+0000","last_fullsized":"2023-11-24T06:24:18.864328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851522+0000","last_change":"2023-11-24T06:24:18.851522+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851522+0000","last_undegraded":"2023-11-24T06:24:18.851522+0000","last_fullsized":"2023-11-24T06:24:18.851522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864303+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:18.864303+0000","last_peered":"2023-11-24T06:24:18.864303+0000","last_clean":"2023-11-24T06:24:18.864303+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:18.864303+0000","last_undegraded":"2023-11-24T06:24:18.864303+0000","last_fullsized":"2023-11-24T06:24:18.864303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843387+0000","last_change":"2023-11-24T06:24:18.843387+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843387+0000","last_undegraded":"2023-11-24T06:24:18.843387+0000","last_fullsized":"2023-11-24T06:24:18.843387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848580+0000","last_change":"2023-11-24T06:24:18.848580+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848580+0000","last_undegraded":"2023-11-24T06:24:18.848580+0000","last_fullsized":"2023-11-24T06:24:18.848580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864891+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:18.864891+0000","last_peered":"2023-11-24T06:24:18.864891+0000","last_clean":"2023-11-24T06:24:18.864891+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:18.864891+0000","last_undegraded":"2023-11-24T06:24:18.864891+0000","last_fullsized":"2023-11-24T06:24:18.864891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845534+0000","last_change":"2023-11-24T06:24:18.845534+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845534+0000","last_undegraded":"2023-11-24T06:24:18.845534+0000","last_fullsized":"2023-11-24T06:24:18.845534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864162+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:18.864162+0000","last_peered":"2023-11-24T06:24:18.864162+0000","last_clean":"2023-11-24T06:24:18.864162+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:18.864162+0000","last_undegraded":"2023-11-24T06:24:18.864162+0000","last_fullsized":"2023-11-24T06:24:18.864162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:24:17.839121+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:17.839121+0000","last_peered":"2023-11-24T06:24:17.839121+0000","last_clean":"2023-11-24T06:24:17.839121+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:17.839121+0000","last_undegraded":"2023-11-24T06:24:17.839121+0000","last_fullsized":"2023-11-24T06:24:17.839121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844436+0000","last_change":"2023-11-24T06:24:18.844436+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844436+0000","last_undegraded":"2023-11-24T06:24:18.844436+0000","last_fullsized":"2023-11-24T06:24:18.844436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861746+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:18.861746+0000","last_peered":"2023-11-24T06:24:18.861746+0000","last_clean":"2023-11-24T06:24:18.861746+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:18.861746+0000","last_undegraded":"2023-11-24T06:24:18.861746+0000","last_fullsized":"2023-11-24T06:24:18.861746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862011+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:18.862011+0000","last_peered":"2023-11-24T06:24:18.862011+0000","last_clean":"2023-11-24T06:24:18.862011+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:18.862011+0000","last_undegraded":"2023-11-24T06:24:18.862011+0000","last_fullsized":"2023-11-24T06:24:18.862011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850677+0000","last_change":"2023-11-24T06:24:18.850677+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850677+0000","last_undegraded":"2023-11-24T06:24:18.850677+0000","last_fullsized":"2023-11-24T06:24:18.850677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861727+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:18.861727+0000","last_peered":"2023-11-24T06:24:18.861727+0000","last_clean":"2023-11-24T06:24:18.861727+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:18.861727+0000","last_undegraded":"2023-11-24T06:24:18.861727+0000","last_fullsized":"2023-11-24T06:24:18.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849533+0000","last_change":"2023-11-24T06:24:18.849533+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849533+0000","last_undegraded":"2023-11-24T06:24:18.849533+0000","last_fullsized":"2023-11-24T06:24:18.849533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864318+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:18.864318+0000","last_peered":"2023-11-24T06:24:18.864318+0000","last_clean":"2023-11-24T06:24:18.864318+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:18.864318+0000","last_undegraded":"2023-11-24T06:24:18.864318+0000","last_fullsized":"2023-11-24T06:24:18.864318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844472+0000","last_change":"2023-11-24T06:24:18.844472+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844472+0000","last_undegraded":"2023-11-24T06:24:18.844472+0000","last_fullsized":"2023-11-24T06:24:18.844472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862045+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:18.862045+0000","last_peered":"2023-11-24T06:24:18.862045+0000","last_clean":"2023-11-24T06:24:18.862045+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:18.862045+0000","last_undegraded":"2023-11-24T06:24:18.862045+0000","last_fullsized":"2023-11-24T06:24:18.862045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843078+0000","last_change":"2023-11-24T06:24:18.843078+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843078+0000","last_undegraded":"2023-11-24T06:24:18.843078+0000","last_fullsized":"2023-11-24T06:24:18.843078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863798+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:18.863798+0000","last_peered":"2023-11-24T06:24:18.863798+0000","last_clean":"2023-11-24T06:24:18.863798+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:18.863798+0000","last_undegraded":"2023-11-24T06:24:18.863798+0000","last_fullsized":"2023-11-24T06:24:18.863798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845239+0000","last_change":"2023-11-24T06:24:18.845239+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845239+0000","last_undegraded":"2023-11-24T06:24:18.845239+0000","last_fullsized":"2023-11-24T06:24:18.845239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864361+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:18.864361+0000","last_peered":"2023-11-24T06:24:18.864361+0000","last_clean":"2023-11-24T06:24:18.864361+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:18.864361+0000","last_undegraded":"2023-11-24T06:24:18.864361+0000","last_fullsized":"2023-11-24T06:24:18.864361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845746+0000","last_change":"2023-11-24T06:24:18.845746+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845746+0000","last_undegraded":"2023-11-24T06:24:18.845746+0000","last_fullsized":"2023-11-24T06:24:18.845746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861865+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:18.861865+0000","last_peered":"2023-11-24T06:24:18.861865+0000","last_clean":"2023-11-24T06:24:18.861865+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:18.861865+0000","last_undegraded":"2023-11-24T06:24:18.861865+0000","last_fullsized":"2023-11-24T06:24:18.861865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850115+0000","last_change":"2023-11-24T06:24:18.850115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850115+0000","last_undegraded":"2023-11-24T06:24:18.850115+0000","last_fullsized":"2023-11-24T06:24:18.850115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865516+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:18.865516+0000","last_peered":"2023-11-24T06:24:18.865516+0000","last_clean":"2023-11-24T06:24:18.865516+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:18.865516+0000","last_undegraded":"2023-11-24T06:24:18.865516+0000","last_fullsized":"2023-11-24T06:24:18.865516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844391+0000","last_change":"2023-11-24T06:24:18.844391+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844391+0000","last_undegraded":"2023-11-24T06:24:18.844391+0000","last_fullsized":"2023-11-24T06:24:18.844391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865112+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:18.865112+0000","last_peered":"2023-11-24T06:24:18.865112+0000","last_clean":"2023-11-24T06:24:18.865112+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:18.865112+0000","last_undegraded":"2023-11-24T06:24:18.865112+0000","last_fullsized":"2023-11-24T06:24:18.865112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854299+0000","last_change":"2023-11-24T06:24:18.854299+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854299+0000","last_undegraded":"2023-11-24T06:24:18.854299+0000","last_fullsized":"2023-11-24T06:24:18.854299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862641+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:18.862641+0000","last_peered":"2023-11-24T06:24:18.862641+0000","last_clean":"2023-11-24T06:24:18.862641+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:18.862641+0000","last_undegraded":"2023-11-24T06:24:18.862641+0000","last_fullsized":"2023-11-24T06:24:18.862641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843946+0000","last_change":"2023-11-24T06:24:18.843946+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843946+0000","last_undegraded":"2023-11-24T06:24:18.843946+0000","last_fullsized":"2023-11-24T06:24:18.843946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854564+0000","last_change":"2023-11-24T06:24:18.854564+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854564+0000","last_undegraded":"2023-11-24T06:24:18.854564+0000","last_fullsized":"2023-11-24T06:24:18.854564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850064+0000","last_change":"2023-11-24T06:24:18.850064+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850064+0000","last_undegraded":"2023-11-24T06:24:18.850064+0000","last_fullsized":"2023-11-24T06:24:18.850064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850695+0000","last_change":"2023-11-24T06:24:18.850695+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850695+0000","last_undegraded":"2023-11-24T06:24:18.850695+0000","last_fullsized":"2023-11-24T06:24:18.850695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845998+0000","last_change":"2023-11-24T06:24:18.845998+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845998+0000","last_undegraded":"2023-11-24T06:24:18.845998+0000","last_fullsized":"2023-11-24T06:24:18.845998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849115+0000","last_change":"2023-11-24T06:24:18.849115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849115+0000","last_undegraded":"2023-11-24T06:24:18.849115+0000","last_fullsized":"2023-11-24T06:24:18.849115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851388+0000","last_change":"2023-11-24T06:24:18.851388+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851388+0000","last_undegraded":"2023-11-24T06:24:18.851388+0000","last_fullsized":"2023-11-24T06:24:18.851388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851520+0000","last_change":"2023-11-24T06:24:18.851520+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851520+0000","last_undegraded":"2023-11-24T06:24:18.851520+0000","last_fullsized":"2023-11-24T06:24:18.851520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849982+0000","last_change":"2023-11-24T06:24:18.849982+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849982+0000","last_undegraded":"2023-11-24T06:24:18.849982+0000","last_fullsized":"2023-11-24T06:24:18.849982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847930+0000","last_change":"2023-11-24T06:24:18.847930+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847930+0000","last_undegraded":"2023-11-24T06:24:18.847930+0000","last_fullsized":"2023-11-24T06:24:18.847930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850083+0000","last_change":"2023-11-24T06:24:18.850083+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850083+0000","last_undegraded":"2023-11-24T06:24:18.850083+0000","last_fullsized":"2023-11-24T06:24:18.850083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851624+0000","last_change":"2023-11-24T06:24:18.851624+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851624+0000","last_undegraded":"2023-11-24T06:24:18.851624+0000","last_fullsized":"2023-11-24T06:24:18.851624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846419+0000","last_change":"2023-11-24T06:24:18.846419+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846419+0000","last_undegraded":"2023-11-24T06:24:18.846419+0000","last_fullsized":"2023-11-24T06:24:18.846419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849459+0000","last_change":"2023-11-24T06:24:18.849459+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849459+0000","last_undegraded":"2023-11-24T06:24:18.849459+0000","last_fullsized":"2023-11-24T06:24:18.849459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":646599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":652874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:24.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:24.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.502+0000 7fdbcf87d700 1 Processor -- start 2023-11-24T06:24:24.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.503+0000 7fdbcf87d700 1 -- start start 2023-11-24T06:24:24.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcf87d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc806c710 0x7fdbc80cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcf87d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbc80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcf87d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80d09a0 con 0x7fdbc8152cc0 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcf87d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc815c4b0 con 0x7fdbc806c710 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcf87d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815c790 con 0x7fdbc80b49a0 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcd619700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdbc8152cc0 0x7fdbc806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53516/0 (socket says 172.21.15.114:53516) 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcd619700 1 -- 172.21.15.114:0/3272202348 learned_addr learned my addr 172.21.15.114:0/3272202348 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:24.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcce18700 1 --2- 172.21.15.114:0/3272202348 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806c710 0x7fdbc80cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcde1a700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbc80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791752834 0 0) 0x7fdbc80d09a0 con 0x7fdbc8152cc0 2023-11-24T06:24:24.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb40035b0 con 0x7fdbc8152cc0 2023-11-24T06:24:24.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543269103 0 0) 0x7fdbb40035b0 con 0x7fdbc8152cc0 2023-11-24T06:24:24.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 msgr2=0x7fdbc80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbc80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806c710 msgr2=0x7fdbc80cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 --2- 172.21.15.114:0/3272202348 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806c710 0x7fdbc80cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc815ca70 con 0x7fdbc8152cc0 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fdbc00019f0 con 0x7fdbc8152cc0 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 >> v1:172.21.15.114:6789/0 conn(0x7fdbc8152cc0 legacy=0x7fdbc806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb4003a00 0x7fdbb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb4004d90 0x7fdbb4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815c4b0 con 0x7fdbb4003a00 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815c790 con 0x7fdbb4004d90 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400ba10 con 0x7fdbc80b49a0 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcd619700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb4003a00 0x7fdbb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcce18700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.504+0000 7fdbcde1a700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb4004d90 0x7fdbb4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 -- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 msgr2=0x7fdbb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc80b49a0 0x7fdbb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 -- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb4004d90 msgr2=0x7fdbb4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb4004d90 0x7fdbb4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 -- 172.21.15.114:0/3272202348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40035b0 con 0x7fdbb4003a00 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcd619700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb4003a00 0x7fdbb4003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000a8d0 tx=0x7fdbb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec49d9bc17c7e8da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000a340 con 0x7fdbb4003a00 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0012600 con 0x7fdbb4003a00 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbbffff700 1 -- 172.21.15.114:0/3272202348 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00129b0 con 0x7fdbb4003a00 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 -- 172.21.15.114:0/3272202348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb4003a00 msgr2=0x7fdbb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/3272202348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb4003a00 0x7fdbb4003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000a8d0 tx=0x7fdbb4005880 comp rx=0 tx=0).stop 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcde1a700 1 -- 172.21.15.114:0/3272202348 reap_dead start 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 -- 172.21.15.114:0/3272202348 shutdown_connections 2023-11-24T06:24:24.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 -- 172.21.15.114:0/3272202348 >> 172.21.15.114:0/3272202348 conn(0x7fdbc8071b90 msgr2=0x7fdbc8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 -- 172.21.15.114:0/3272202348 shutdown_connections 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.505+0000 7fdbcf87d700 1 -- 172.21.15.114:0/3272202348 wait complete. 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 Processor -- start 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 -- start start 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc8147130 0x7fdbc814a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc8153820 0x7fdbc806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc814e6c0 con 0x7fdbc81533a0 2023-11-24T06:24:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8147af0 con 0x7fdbc8153820 2023-11-24T06:24:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcf87d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc814bb50 con 0x7fdbc8147130 2023-11-24T06:24:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcce18700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcce18700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37822/0 (socket says 172.21.15.114:37822) 2023-11-24T06:24:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcce18700 1 -- 172.21.15.114:0/2391876752 learned_addr learned my addr 172.21.15.114:0/2391876752 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcd619700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc8147130 0x7fdbc814a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.507+0000 7fdbcde1a700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc8153820 0x7fdbc806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 -- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc8147130 msgr2=0x7fdbc814a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc8147130 0x7fdbc814a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 -- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc8153820 msgr2=0x7fdbc806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc8153820 0x7fdbc806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 -- 172.21.15.114:0/2391876752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc814be30 con 0x7fdbc81533a0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcce18700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8011ad0 tx=0x7fdbb8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800f940 con 0x7fdbc81533a0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb800faa0 con 0x7fdbc81533a0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc814c110 con 0x7fdbc81533a0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800fe50 con 0x7fdbc81533a0 2023-11-24T06:24:24.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.508+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc814c6d0 con 0x7fdbc81533a0 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.509+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc80753f0 con 0x7fdbc81533a0 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.510+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbb801d400 con 0x7fdbc81533a0 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.510+0000 7fdbbdffb700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 0x7fdbac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.510+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbb8089b60 con 0x7fdbc81533a0 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.510+0000 7fdbcd619700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 0x7fdbac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.510+0000 7fdbcd619700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 0x7fdbac06d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b4130 tx=0x7fdbc0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.514+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbb8054c10 con 0x7fdbc81533a0 2023-11-24T06:24:24.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.698+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbc80d0460 con 0x7fdbac06aed0 2023-11-24T06:24:24.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.705+0000 7fdbbdffb700 1 -- 172.21.15.114:0/2391876752 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259383 (secure 0 0 0) 0x7fdbc0022070 con 0x7fdbac06aed0 2023-11-24T06:24:24.711 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.707+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 msgr2=0x7fdbac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.707+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 0x7fdbac06d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b4130 tx=0x7fdbc0009030 comp rx=0 tx=0).stop 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 msgr2=0x7fdbc806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8011ad0 tx=0x7fdbb8011de0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 shutdown_connections 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbac06aed0 0x7fdbac06d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc8153820 0x7fdbc806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbc8147130 0x7fdbc814a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 --2- 172.21.15.114:0/2391876752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc81533a0 0x7fdbc806b7c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 >> 172.21.15.114:0/2391876752 conn(0x7fdbc8071b90 msgr2=0x7fdbc80adf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:24.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 shutdown_connections 2023-11-24T06:24:24.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.708+0000 7fdbcf87d700 1 -- 172.21.15.114:0/2391876752 wait complete. 2023-11-24T06:24:24.714 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:24.723 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:24:23.708302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2703360,"data_stored":1668234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845901+0000","last_change":"2023-11-24T06:24:18.845901+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845901+0000","last_undegraded":"2023-11-24T06:24:18.845901+0000","last_fullsized":"2023-11-24T06:24:18.845901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851607+0000","last_change":"2023-11-24T06:24:18.851607+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851607+0000","last_undegraded":"2023-11-24T06:24:18.851607+0000","last_fullsized":"2023-11-24T06:24:18.851607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846763+0000","last_change":"2023-11-24T06:24:18.846763+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846763+0000","last_undegraded":"2023-11-24T06:24:18.846763+0000","last_fullsized":"2023-11-24T06:24:18.846763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850784+0000","last_change":"2023-11-24T06:24:18.850784+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850784+0000","last_undegraded":"2023-11-24T06:24:18.850784+0000","last_fullsized":"2023-11-24T06:24:18.850784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844740+0000","last_change":"2023-11-24T06:24:18.844740+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844740+0000","last_undegraded":"2023-11-24T06:24:18.844740+0000","last_fullsized":"2023-11-24T06:24:18.844740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.853282+0000","last_change":"2023-11-24T06:24:18.853282+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.853282+0000","last_undegraded":"2023-11-24T06:24:18.853282+0000","last_fullsized":"2023-11-24T06:24:18.853282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845753+0000","last_change":"2023-11-24T06:24:18.845753+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845753+0000","last_undegraded":"2023-11-24T06:24:18.845753+0000","last_fullsized":"2023-11-24T06:24:18.845753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851451+0000","last_change":"2023-11-24T06:24:18.851451+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851451+0000","last_undegraded":"2023-11-24T06:24:18.851451+0000","last_fullsized":"2023-11-24T06:24:18.851451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845084+0000","last_change":"2023-11-24T06:24:18.845084+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845084+0000","last_undegraded":"2023-11-24T06:24:18.845084+0000","last_fullsized":"2023-11-24T06:24:18.845084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849004+0000","last_change":"2023-11-24T06:24:18.849004+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849004+0000","last_undegraded":"2023-11-24T06:24:18.849004+0000","last_fullsized":"2023-11-24T06:24:18.849004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850853+0000","last_change":"2023-11-24T06:24:18.850853+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850853+0000","last_undegraded":"2023-11-24T06:24:18.850853+0000","last_fullsized":"2023-11-24T06:24:18.850853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844967+0000","last_change":"2023-11-24T06:24:18.844967+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844967+0000","last_undegraded":"2023-11-24T06:24:18.844967+0000","last_fullsized":"2023-11-24T06:24:18.844967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851767+0000","last_change":"2023-11-24T06:24:18.851767+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851767+0000","last_undegraded":"2023-11-24T06:24:18.851767+0000","last_fullsized":"2023-11-24T06:24:18.851767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846586+0000","last_change":"2023-11-24T06:24:18.846586+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846586+0000","last_undegraded":"2023-11-24T06:24:18.846586+0000","last_fullsized":"2023-11-24T06:24:18.846586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851597+0000","last_change":"2023-11-24T06:24:18.851597+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851597+0000","last_undegraded":"2023-11-24T06:24:18.851597+0000","last_fullsized":"2023-11-24T06:24:18.851597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851790+0000","last_change":"2023-11-24T06:24:18.851790+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851790+0000","last_undegraded":"2023-11-24T06:24:18.851790+0000","last_fullsized":"2023-11-24T06:24:18.851790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848814+0000","last_change":"2023-11-24T06:24:18.848814+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848814+0000","last_undegraded":"2023-11-24T06:24:18.848814+0000","last_fullsized":"2023-11-24T06:24:18.848814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850442+0000","last_change":"2023-11-24T06:24:18.850442+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850442+0000","last_undegraded":"2023-11-24T06:24:18.850442+0000","last_fullsized":"2023-11-24T06:24:18.850442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846097+0000","last_change":"2023-11-24T06:24:18.846097+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846097+0000","last_undegraded":"2023-11-24T06:24:18.846097+0000","last_fullsized":"2023-11-24T06:24:18.846097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859398+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:18.859398+0000","last_peered":"2023-11-24T06:24:18.859398+0000","last_clean":"2023-11-24T06:24:18.859398+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:18.859398+0000","last_undegraded":"2023-11-24T06:24:18.859398+0000","last_fullsized":"2023-11-24T06:24:18.859398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849020+0000","last_change":"2023-11-24T06:24:18.849020+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849020+0000","last_undegraded":"2023-11-24T06:24:18.849020+0000","last_fullsized":"2023-11-24T06:24:18.849020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859724+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:18.859724+0000","last_peered":"2023-11-24T06:24:18.859724+0000","last_clean":"2023-11-24T06:24:18.859724+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:18.859724+0000","last_undegraded":"2023-11-24T06:24:18.859724+0000","last_fullsized":"2023-11-24T06:24:18.859724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847546+0000","last_change":"2023-11-24T06:24:18.847546+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847546+0000","last_undegraded":"2023-11-24T06:24:18.847546+0000","last_fullsized":"2023-11-24T06:24:18.847546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862355+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:18.862355+0000","last_peered":"2023-11-24T06:24:18.862355+0000","last_clean":"2023-11-24T06:24:18.862355+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:18.862355+0000","last_undegraded":"2023-11-24T06:24:18.862355+0000","last_fullsized":"2023-11-24T06:24:18.862355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.856218+0000","last_change":"2023-11-24T06:24:18.856218+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.856218+0000","last_undegraded":"2023-11-24T06:24:18.856218+0000","last_fullsized":"2023-11-24T06:24:18.856218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864140+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:18.864140+0000","last_peered":"2023-11-24T06:24:18.864140+0000","last_clean":"2023-11-24T06:24:18.864140+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:18.864140+0000","last_undegraded":"2023-11-24T06:24:18.864140+0000","last_fullsized":"2023-11-24T06:24:18.864140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850116+0000","last_change":"2023-11-24T06:24:18.850116+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850116+0000","last_undegraded":"2023-11-24T06:24:18.850116+0000","last_fullsized":"2023-11-24T06:24:18.850116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865327+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:18.865327+0000","last_peered":"2023-11-24T06:24:18.865327+0000","last_clean":"2023-11-24T06:24:18.865327+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:18.865327+0000","last_undegraded":"2023-11-24T06:24:18.865327+0000","last_fullsized":"2023-11-24T06:24:18.865327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843310+0000","last_change":"2023-11-24T06:24:18.843310+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843310+0000","last_undegraded":"2023-11-24T06:24:18.843310+0000","last_fullsized":"2023-11-24T06:24:18.843310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859116+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:18.859116+0000","last_peered":"2023-11-24T06:24:18.859116+0000","last_clean":"2023-11-24T06:24:18.859116+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:18.859116+0000","last_undegraded":"2023-11-24T06:24:18.859116+0000","last_fullsized":"2023-11-24T06:24:18.859116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845473+0000","last_change":"2023-11-24T06:24:18.845473+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845473+0000","last_undegraded":"2023-11-24T06:24:18.845473+0000","last_fullsized":"2023-11-24T06:24:18.845473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859717+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:18.859717+0000","last_peered":"2023-11-24T06:24:18.859717+0000","last_clean":"2023-11-24T06:24:18.859717+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:18.859717+0000","last_undegraded":"2023-11-24T06:24:18.859717+0000","last_fullsized":"2023-11-24T06:24:18.859717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845987+0000","last_change":"2023-11-24T06:24:18.845987+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845987+0000","last_undegraded":"2023-11-24T06:24:18.845987+0000","last_fullsized":"2023-11-24T06:24:18.845987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863686+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:18.863686+0000","last_peered":"2023-11-24T06:24:18.863686+0000","last_clean":"2023-11-24T06:24:18.863686+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:18.863686+0000","last_undegraded":"2023-11-24T06:24:18.863686+0000","last_fullsized":"2023-11-24T06:24:18.863686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850409+0000","last_change":"2023-11-24T06:24:18.850409+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850409+0000","last_undegraded":"2023-11-24T06:24:18.850409+0000","last_fullsized":"2023-11-24T06:24:18.850409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864305+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:18.864305+0000","last_peered":"2023-11-24T06:24:18.864305+0000","last_clean":"2023-11-24T06:24:18.864305+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:18.864305+0000","last_undegraded":"2023-11-24T06:24:18.864305+0000","last_fullsized":"2023-11-24T06:24:18.864305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850760+0000","last_change":"2023-11-24T06:24:18.850760+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850760+0000","last_undegraded":"2023-11-24T06:24:18.850760+0000","last_fullsized":"2023-11-24T06:24:18.850760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863658+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:18.863658+0000","last_peered":"2023-11-24T06:24:18.863658+0000","last_clean":"2023-11-24T06:24:18.863658+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:18.863658+0000","last_undegraded":"2023-11-24T06:24:18.863658+0000","last_fullsized":"2023-11-24T06:24:18.863658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847096+0000","last_change":"2023-11-24T06:24:18.847096+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847096+0000","last_undegraded":"2023-11-24T06:24:18.847096+0000","last_fullsized":"2023-11-24T06:24:18.847096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.866421+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:18.866421+0000","last_peered":"2023-11-24T06:24:18.866421+0000","last_clean":"2023-11-24T06:24:18.866421+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:18.866421+0000","last_undegraded":"2023-11-24T06:24:18.866421+0000","last_fullsized":"2023-11-24T06:24:18.866421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843113+0000","last_change":"2023-11-24T06:24:18.843113+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843113+0000","last_undegraded":"2023-11-24T06:24:18.843113+0000","last_fullsized":"2023-11-24T06:24:18.843113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863776+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:18.863776+0000","last_peered":"2023-11-24T06:24:18.863776+0000","last_clean":"2023-11-24T06:24:18.863776+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:18.863776+0000","last_undegraded":"2023-11-24T06:24:18.863776+0000","last_fullsized":"2023-11-24T06:24:18.863776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845117+0000","last_change":"2023-11-24T06:24:18.845117+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845117+0000","last_undegraded":"2023-11-24T06:24:18.845117+0000","last_fullsized":"2023-11-24T06:24:18.845117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.856003+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:18.856003+0000","last_peered":"2023-11-24T06:24:18.856003+0000","last_clean":"2023-11-24T06:24:18.856003+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:18.856003+0000","last_undegraded":"2023-11-24T06:24:18.856003+0000","last_fullsized":"2023-11-24T06:24:18.856003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845031+0000","last_change":"2023-11-24T06:24:18.845031+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845031+0000","last_undegraded":"2023-11-24T06:24:18.845031+0000","last_fullsized":"2023-11-24T06:24:18.845031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863773+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:18.863773+0000","last_peered":"2023-11-24T06:24:18.863773+0000","last_clean":"2023-11-24T06:24:18.863773+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:18.863773+0000","last_undegraded":"2023-11-24T06:24:18.863773+0000","last_fullsized":"2023-11-24T06:24:18.863773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843918+0000","last_change":"2023-11-24T06:24:18.843918+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843918+0000","last_undegraded":"2023-11-24T06:24:18.843918+0000","last_fullsized":"2023-11-24T06:24:18.843918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859552+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:18.859552+0000","last_peered":"2023-11-24T06:24:18.859552+0000","last_clean":"2023-11-24T06:24:18.859552+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:18.859552+0000","last_undegraded":"2023-11-24T06:24:18.859552+0000","last_fullsized":"2023-11-24T06:24:18.859552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849274+0000","last_change":"2023-11-24T06:24:18.849274+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849274+0000","last_undegraded":"2023-11-24T06:24:18.849274+0000","last_fullsized":"2023-11-24T06:24:18.849274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865485+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:18.865485+0000","last_peered":"2023-11-24T06:24:18.865485+0000","last_clean":"2023-11-24T06:24:18.865485+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:18.865485+0000","last_undegraded":"2023-11-24T06:24:18.865485+0000","last_fullsized":"2023-11-24T06:24:18.865485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844048+0000","last_change":"2023-11-24T06:24:18.844048+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844048+0000","last_undegraded":"2023-11-24T06:24:18.844048+0000","last_fullsized":"2023-11-24T06:24:18.844048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.858819+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:18.858819+0000","last_peered":"2023-11-24T06:24:18.858819+0000","last_clean":"2023-11-24T06:24:18.858819+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:18.858819+0000","last_undegraded":"2023-11-24T06:24:18.858819+0000","last_fullsized":"2023-11-24T06:24:18.858819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844498+0000","last_change":"2023-11-24T06:24:18.844498+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844498+0000","last_undegraded":"2023-11-24T06:24:18.844498+0000","last_fullsized":"2023-11-24T06:24:18.844498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864328+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:18.864328+0000","last_peered":"2023-11-24T06:24:18.864328+0000","last_clean":"2023-11-24T06:24:18.864328+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:18.864328+0000","last_undegraded":"2023-11-24T06:24:18.864328+0000","last_fullsized":"2023-11-24T06:24:18.864328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851522+0000","last_change":"2023-11-24T06:24:18.851522+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851522+0000","last_undegraded":"2023-11-24T06:24:18.851522+0000","last_fullsized":"2023-11-24T06:24:18.851522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864303+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:18.864303+0000","last_peered":"2023-11-24T06:24:18.864303+0000","last_clean":"2023-11-24T06:24:18.864303+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:18.864303+0000","last_undegraded":"2023-11-24T06:24:18.864303+0000","last_fullsized":"2023-11-24T06:24:18.864303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843387+0000","last_change":"2023-11-24T06:24:18.843387+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843387+0000","last_undegraded":"2023-11-24T06:24:18.843387+0000","last_fullsized":"2023-11-24T06:24:18.843387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848580+0000","last_change":"2023-11-24T06:24:18.848580+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848580+0000","last_undegraded":"2023-11-24T06:24:18.848580+0000","last_fullsized":"2023-11-24T06:24:18.848580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864891+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:18.864891+0000","last_peered":"2023-11-24T06:24:18.864891+0000","last_clean":"2023-11-24T06:24:18.864891+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:18.864891+0000","last_undegraded":"2023-11-24T06:24:18.864891+0000","last_fullsized":"2023-11-24T06:24:18.864891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845534+0000","last_change":"2023-11-24T06:24:18.845534+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845534+0000","last_undegraded":"2023-11-24T06:24:18.845534+0000","last_fullsized":"2023-11-24T06:24:18.845534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864162+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:18.864162+0000","last_peered":"2023-11-24T06:24:18.864162+0000","last_clean":"2023-11-24T06:24:18.864162+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:18.864162+0000","last_undegraded":"2023-11-24T06:24:18.864162+0000","last_fullsized":"2023-11-24T06:24:18.864162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:24:17.839121+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:17.839121+0000","last_peered":"2023-11-24T06:24:17.839121+0000","last_clean":"2023-11-24T06:24:17.839121+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:17.839121+0000","last_undegraded":"2023-11-24T06:24:17.839121+0000","last_fullsized":"2023-11-24T06:24:17.839121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844436+0000","last_change":"2023-11-24T06:24:18.844436+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844436+0000","last_undegraded":"2023-11-24T06:24:18.844436+0000","last_fullsized":"2023-11-24T06:24:18.844436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861746+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:18.861746+0000","last_peered":"2023-11-24T06:24:18.861746+0000","last_clean":"2023-11-24T06:24:18.861746+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:18.861746+0000","last_undegraded":"2023-11-24T06:24:18.861746+0000","last_fullsized":"2023-11-24T06:24:18.861746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862011+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:18.862011+0000","last_peered":"2023-11-24T06:24:18.862011+0000","last_clean":"2023-11-24T06:24:18.862011+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:18.862011+0000","last_undegraded":"2023-11-24T06:24:18.862011+0000","last_fullsized":"2023-11-24T06:24:18.862011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850677+0000","last_change":"2023-11-24T06:24:18.850677+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850677+0000","last_undegraded":"2023-11-24T06:24:18.850677+0000","last_fullsized":"2023-11-24T06:24:18.850677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861727+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:18.861727+0000","last_peered":"2023-11-24T06:24:18.861727+0000","last_clean":"2023-11-24T06:24:18.861727+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:18.861727+0000","last_undegraded":"2023-11-24T06:24:18.861727+0000","last_fullsized":"2023-11-24T06:24:18.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849533+0000","last_change":"2023-11-24T06:24:18.849533+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849533+0000","last_undegraded":"2023-11-24T06:24:18.849533+0000","last_fullsized":"2023-11-24T06:24:18.849533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864318+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:18.864318+0000","last_peered":"2023-11-24T06:24:18.864318+0000","last_clean":"2023-11-24T06:24:18.864318+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:18.864318+0000","last_undegraded":"2023-11-24T06:24:18.864318+0000","last_fullsized":"2023-11-24T06:24:18.864318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844472+0000","last_change":"2023-11-24T06:24:18.844472+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844472+0000","last_undegraded":"2023-11-24T06:24:18.844472+0000","last_fullsized":"2023-11-24T06:24:18.844472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862045+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:18.862045+0000","last_peered":"2023-11-24T06:24:18.862045+0000","last_clean":"2023-11-24T06:24:18.862045+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:18.862045+0000","last_undegraded":"2023-11-24T06:24:18.862045+0000","last_fullsized":"2023-11-24T06:24:18.862045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843078+0000","last_change":"2023-11-24T06:24:18.843078+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843078+0000","last_undegraded":"2023-11-24T06:24:18.843078+0000","last_fullsized":"2023-11-24T06:24:18.843078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863798+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:18.863798+0000","last_peered":"2023-11-24T06:24:18.863798+0000","last_clean":"2023-11-24T06:24:18.863798+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:18.863798+0000","last_undegraded":"2023-11-24T06:24:18.863798+0000","last_fullsized":"2023-11-24T06:24:18.863798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845239+0000","last_change":"2023-11-24T06:24:18.845239+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845239+0000","last_undegraded":"2023-11-24T06:24:18.845239+0000","last_fullsized":"2023-11-24T06:24:18.845239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864361+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:18.864361+0000","last_peered":"2023-11-24T06:24:18.864361+0000","last_clean":"2023-11-24T06:24:18.864361+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:18.864361+0000","last_undegraded":"2023-11-24T06:24:18.864361+0000","last_fullsized":"2023-11-24T06:24:18.864361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845746+0000","last_change":"2023-11-24T06:24:18.845746+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845746+0000","last_undegraded":"2023-11-24T06:24:18.845746+0000","last_fullsized":"2023-11-24T06:24:18.845746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861865+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:18.861865+0000","last_peered":"2023-11-24T06:24:18.861865+0000","last_clean":"2023-11-24T06:24:18.861865+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:18.861865+0000","last_undegraded":"2023-11-24T06:24:18.861865+0000","last_fullsized":"2023-11-24T06:24:18.861865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850115+0000","last_change":"2023-11-24T06:24:18.850115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850115+0000","last_undegraded":"2023-11-24T06:24:18.850115+0000","last_fullsized":"2023-11-24T06:24:18.850115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865516+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:18.865516+0000","last_peered":"2023-11-24T06:24:18.865516+0000","last_clean":"2023-11-24T06:24:18.865516+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:18.865516+0000","last_undegraded":"2023-11-24T06:24:18.865516+0000","last_fullsized":"2023-11-24T06:24:18.865516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844391+0000","last_change":"2023-11-24T06:24:18.844391+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844391+0000","last_undegraded":"2023-11-24T06:24:18.844391+0000","last_fullsized":"2023-11-24T06:24:18.844391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865112+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:18.865112+0000","last_peered":"2023-11-24T06:24:18.865112+0000","last_clean":"2023-11-24T06:24:18.865112+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:18.865112+0000","last_undegraded":"2023-11-24T06:24:18.865112+0000","last_fullsized":"2023-11-24T06:24:18.865112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854299+0000","last_change":"2023-11-24T06:24:18.854299+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854299+0000","last_undegraded":"2023-11-24T06:24:18.854299+0000","last_fullsized":"2023-11-24T06:24:18.854299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862641+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:18.862641+0000","last_peered":"2023-11-24T06:24:18.862641+0000","last_clean":"2023-11-24T06:24:18.862641+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:18.862641+0000","last_undegraded":"2023-11-24T06:24:18.862641+0000","last_fullsized":"2023-11-24T06:24:18.862641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843946+0000","last_change":"2023-11-24T06:24:18.843946+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843946+0000","last_undegraded":"2023-11-24T06:24:18.843946+0000","last_fullsized":"2023-11-24T06:24:18.843946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854564+0000","last_change":"2023-11-24T06:24:18.854564+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854564+0000","last_undegraded":"2023-11-24T06:24:18.854564+0000","last_fullsized":"2023-11-24T06:24:18.854564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850064+0000","last_change":"2023-11-24T06:24:18.850064+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850064+0000","last_undegraded":"2023-11-24T06:24:18.850064+0000","last_fullsized":"2023-11-24T06:24:18.850064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850695+0000","last_change":"2023-11-24T06:24:18.850695+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850695+0000","last_undegraded":"2023-11-24T06:24:18.850695+0000","last_fullsized":"2023-11-24T06:24:18.850695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845998+0000","last_change":"2023-11-24T06:24:18.845998+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845998+0000","last_undegraded":"2023-11-24T06:24:18.845998+0000","last_fullsized":"2023-11-24T06:24:18.845998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849115+0000","last_change":"2023-11-24T06:24:18.849115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849115+0000","last_undegraded":"2023-11-24T06:24:18.849115+0000","last_fullsized":"2023-11-24T06:24:18.849115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851388+0000","last_change":"2023-11-24T06:24:18.851388+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851388+0000","last_undegraded":"2023-11-24T06:24:18.851388+0000","last_fullsized":"2023-11-24T06:24:18.851388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851520+0000","last_change":"2023-11-24T06:24:18.851520+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851520+0000","last_undegraded":"2023-11-24T06:24:18.851520+0000","last_fullsized":"2023-11-24T06:24:18.851520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849982+0000","last_change":"2023-11-24T06:24:18.849982+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849982+0000","last_undegraded":"2023-11-24T06:24:18.849982+0000","last_fullsized":"2023-11-24T06:24:18.849982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847930+0000","last_change":"2023-11-24T06:24:18.847930+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847930+0000","last_undegraded":"2023-11-24T06:24:18.847930+0000","last_fullsized":"2023-11-24T06:24:18.847930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850083+0000","last_change":"2023-11-24T06:24:18.850083+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850083+0000","last_undegraded":"2023-11-24T06:24:18.850083+0000","last_fullsized":"2023-11-24T06:24:18.850083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851624+0000","last_change":"2023-11-24T06:24:18.851624+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851624+0000","last_undegraded":"2023-11-24T06:24:18.851624+0000","last_fullsized":"2023-11-24T06:24:18.851624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846419+0000","last_change":"2023-11-24T06:24:18.846419+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846419+0000","last_undegraded":"2023-11-24T06:24:18.846419+0000","last_fullsized":"2023-11-24T06:24:18.846419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849459+0000","last_change":"2023-11-24T06:24:18.849459+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849459+0000","last_undegraded":"2023-11-24T06:24:18.849459+0000","last_fullsized":"2023-11-24T06:24:18.849459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":646599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":652874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:24.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845901+0000', 'last_change': '2023-11-24T06:24:18.845901+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845901+0000', 'last_undegraded': '2023-11-24T06:24:18.845901+0000', 'last_fullsized': '2023-11-24T06:24:18.845901+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851607+0000', 'last_change': '2023-11-24T06:24:18.851607+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851607+0000', 'last_undegraded': '2023-11-24T06:24:18.851607+0000', 'last_fullsized': '2023-11-24T06:24:18.851607+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.846763+0000', 'last_change': '2023-11-24T06:24:18.846763+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.846763+0000', 'last_undegraded': '2023-11-24T06:24:18.846763+0000', 'last_fullsized': '2023-11-24T06:24:18.846763+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850784+0000', 'last_change': '2023-11-24T06:24:18.850784+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850784+0000', 'last_undegraded': '2023-11-24T06:24:18.850784+0000', 'last_fullsized': '2023-11-24T06:24:18.850784+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-11-24T06:24:24.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844740+0000', 'last_change': '2023-11-24T06:24:18.844740+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844740+0000', 'last_undegraded': '2023-11-24T06:24:18.844740+0000', 'last_fullsized': '2023-11-24T06:24:18.844740+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.853282+0000', 'last_change': '2023-11-24T06:24:18.853282+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.853282+0000', 'last_undegraded': '2023-11-24T06:24:18.853282+0000', 'last_fullsized': '2023-11-24T06:24:18.853282+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845753+0000', 'last_change': '2023-11-24T06:24:18.845753+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845753+0000', 'last_undegraded': '2023-11-24T06:24:18.845753+0000', 'last_fullsized': '2023-11-24T06:24:18.845753+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851451+0000', 'last_change': '2023-11-24T06:24:18.851451+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851451+0000', 'last_undegraded': '2023-11-24T06:24:18.851451+0000', 'last_fullsized': '2023-11-24T06:24:18.851451+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845084+0000', 'last_change': '2023-11-24T06:24:18.845084+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845084+0000', 'last_undegraded': '2023-11-24T06:24:18.845084+0000', 'last_fullsized': '2023-11-24T06:24:18.845084+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849004+0000', 'last_change': '2023-11-24T06:24:18.849004+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849004+0000', 'last_undegraded': '2023-11-24T06:24:18.849004+0000', 'last_fullsized': '2023-11-24T06:24:18.849004+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850853+0000', 'last_change': '2023-11-24T06:24:18.850853+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850853+0000', 'last_undegraded': '2023-11-24T06:24:18.850853+0000', 'last_fullsized': '2023-11-24T06:24:18.850853+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844967+0000', 'last_change': '2023-11-24T06:24:18.844967+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844967+0000', 'last_undegraded': '2023-11-24T06:24:18.844967+0000', 'last_fullsized': '2023-11-24T06:24:18.844967+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851767+0000', 'last_change': '2023-11-24T06:24:18.851767+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851767+0000', 'last_undegraded': '2023-11-24T06:24:18.851767+0000', 'last_fullsized': '2023-11-24T06:24:18.851767+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.846586+0000', 'last_change': '2023-11-24T06:24:18.846586+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.846586+0000', 'last_undegraded': '2023-11-24T06:24:18.846586+0000', 'last_fullsized': '2023-11-24T06:24:18.846586+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851597+0000', 'last_change': '2023-11-24T06:24:18.851597+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851597+0000', 'last_undegraded': '2023-11-24T06:24:18.851597+0000', 'last_fullsized': '2023-11-24T06:24:18.851597+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851790+0000', 'last_change': '2023-11-24T06:24:18.851790+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851790+0000', 'last_undegraded': '2023-11-24T06:24:18.851790+0000', 'last_fullsized': '2023-11-24T06:24:18.851790+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.848814+0000', 'last_change': '2023-11-24T06:24:18.848814+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.848814+0000', 'last_undegraded': '2023-11-24T06:24:18.848814+0000', 'last_fullsized': '2023-11-24T06:24:18.848814+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850442+0000', 'last_change': '2023-11-24T06:24:18.850442+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850442+0000', 'last_undegraded': '2023-11-24T06:24:18.850442+0000', 'last_fullsized': '2023-11-24T06:24:18.850442+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.846097+0000', 'last_change': '2023-11-24T06:24:18.846097+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.846097+0000', 'last_undegraded': '2023-11-24T06:24:18.846097+0000', 'last_fullsized': '2023-11-24T06:24:18.846097+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849020+0000', 'last_change': '2023-11-24T06:24:18.849020+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849020+0000', 'last_undegraded': '2023-11-24T06:24:18.849020+0000', 'last_fullsized': '2023-11-24T06:24:18.849020+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-11-24T06:24:24.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.847546+0000', 'last_change': '2023-11-24T06:24:18.847546+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.847546+0000', 'last_undegraded': '2023-11-24T06:24:18.847546+0000', 'last_fullsized': '2023-11-24T06:24:18.847546+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.856218+0000', 'last_change': '2023-11-24T06:24:18.856218+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.856218+0000', 'last_undegraded': '2023-11-24T06:24:18.856218+0000', 'last_fullsized': '2023-11-24T06:24:18.856218+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850116+0000', 'last_change': '2023-11-24T06:24:18.850116+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850116+0000', 'last_undegraded': '2023-11-24T06:24:18.850116+0000', 'last_fullsized': '2023-11-24T06:24:18.850116+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843310+0000', 'last_change': '2023-11-24T06:24:18.843310+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843310+0000', 'last_undegraded': '2023-11-24T06:24:18.843310+0000', 'last_fullsized': '2023-11-24T06:24:18.843310+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845473+0000', 'last_change': '2023-11-24T06:24:18.845473+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845473+0000', 'last_undegraded': '2023-11-24T06:24:18.845473+0000', 'last_fullsized': '2023-11-24T06:24:18.845473+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-11-24T06:24:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845987+0000', 'last_change': '2023-11-24T06:24:18.845987+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845987+0000', 'last_undegraded': '2023-11-24T06:24:18.845987+0000', 'last_fullsized': '2023-11-24T06:24:18.845987+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850409+0000', 'last_change': '2023-11-24T06:24:18.850409+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850409+0000', 'last_undegraded': '2023-11-24T06:24:18.850409+0000', 'last_fullsized': '2023-11-24T06:24:18.850409+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850760+0000', 'last_change': '2023-11-24T06:24:18.850760+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850760+0000', 'last_undegraded': '2023-11-24T06:24:18.850760+0000', 'last_fullsized': '2023-11-24T06:24:18.850760+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.847096+0000', 'last_change': '2023-11-24T06:24:18.847096+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.847096+0000', 'last_undegraded': '2023-11-24T06:24:18.847096+0000', 'last_fullsized': '2023-11-24T06:24:18.847096+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843113+0000', 'last_change': '2023-11-24T06:24:18.843113+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843113+0000', 'last_undegraded': '2023-11-24T06:24:18.843113+0000', 'last_fullsized': '2023-11-24T06:24:18.843113+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845117+0000', 'last_change': '2023-11-24T06:24:18.845117+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845117+0000', 'last_undegraded': '2023-11-24T06:24:18.845117+0000', 'last_fullsized': '2023-11-24T06:24:18.845117+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845031+0000', 'last_change': '2023-11-24T06:24:18.845031+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845031+0000', 'last_undegraded': '2023-11-24T06:24:18.845031+0000', 'last_fullsized': '2023-11-24T06:24:18.845031+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843918+0000', 'last_change': '2023-11-24T06:24:18.843918+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843918+0000', 'last_undegraded': '2023-11-24T06:24:18.843918+0000', 'last_fullsized': '2023-11-24T06:24:18.843918+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849274+0000', 'last_change': '2023-11-24T06:24:18.849274+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849274+0000', 'last_undegraded': '2023-11-24T06:24:18.849274+0000', 'last_fullsized': '2023-11-24T06:24:18.849274+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844048+0000', 'last_change': '2023-11-24T06:24:18.844048+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844048+0000', 'last_undegraded': '2023-11-24T06:24:18.844048+0000', 'last_fullsized': '2023-11-24T06:24:18.844048+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844498+0000', 'last_change': '2023-11-24T06:24:18.844498+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844498+0000', 'last_undegraded': '2023-11-24T06:24:18.844498+0000', 'last_fullsized': '2023-11-24T06:24:18.844498+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851522+0000', 'last_change': '2023-11-24T06:24:18.851522+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851522+0000', 'last_undegraded': '2023-11-24T06:24:18.851522+0000', 'last_fullsized': '2023-11-24T06:24:18.851522+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843387+0000', 'last_change': '2023-11-24T06:24:18.843387+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843387+0000', 'last_undegraded': '2023-11-24T06:24:18.843387+0000', 'last_fullsized': '2023-11-24T06:24:18.843387+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.848580+0000', 'last_change': '2023-11-24T06:24:18.848580+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.848580+0000', 'last_undegraded': '2023-11-24T06:24:18.848580+0000', 'last_fullsized': '2023-11-24T06:24:18.848580+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845534+0000', 'last_change': '2023-11-24T06:24:18.845534+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845534+0000', 'last_undegraded': '2023-11-24T06:24:18.845534+0000', 'last_fullsized': '2023-11-24T06:24:18.845534+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844436+0000', 'last_change': '2023-11-24T06:24:18.844436+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844436+0000', 'last_undegraded': '2023-11-24T06:24:18.844436+0000', 'last_fullsized': '2023-11-24T06:24:18.844436+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-11-24T06:24:24.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850677+0000', 'last_change': '2023-11-24T06:24:18.850677+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850677+0000', 'last_undegraded': '2023-11-24T06:24:18.850677+0000', 'last_fullsized': '2023-11-24T06:24:18.850677+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849533+0000', 'last_change': '2023-11-24T06:24:18.849533+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849533+0000', 'last_undegraded': '2023-11-24T06:24:18.849533+0000', 'last_fullsized': '2023-11-24T06:24:18.849533+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844472+0000', 'last_change': '2023-11-24T06:24:18.844472+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844472+0000', 'last_undegraded': '2023-11-24T06:24:18.844472+0000', 'last_fullsized': '2023-11-24T06:24:18.844472+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843078+0000', 'last_change': '2023-11-24T06:24:18.843078+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843078+0000', 'last_undegraded': '2023-11-24T06:24:18.843078+0000', 'last_fullsized': '2023-11-24T06:24:18.843078+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845239+0000', 'last_change': '2023-11-24T06:24:18.845239+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845239+0000', 'last_undegraded': '2023-11-24T06:24:18.845239+0000', 'last_fullsized': '2023-11-24T06:24:18.845239+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845746+0000', 'last_change': '2023-11-24T06:24:18.845746+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845746+0000', 'last_undegraded': '2023-11-24T06:24:18.845746+0000', 'last_fullsized': '2023-11-24T06:24:18.845746+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850115+0000', 'last_change': '2023-11-24T06:24:18.850115+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850115+0000', 'last_undegraded': '2023-11-24T06:24:18.850115+0000', 'last_fullsized': '2023-11-24T06:24:18.850115+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.844391+0000', 'last_change': '2023-11-24T06:24:18.844391+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.844391+0000', 'last_undegraded': '2023-11-24T06:24:18.844391+0000', 'last_fullsized': '2023-11-24T06:24:18.844391+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.854299+0000', 'last_change': '2023-11-24T06:24:18.854299+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.854299+0000', 'last_undegraded': '2023-11-24T06:24:18.854299+0000', 'last_fullsized': '2023-11-24T06:24:18.854299+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.843946+0000', 'last_change': '2023-11-24T06:24:18.843946+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.843946+0000', 'last_undegraded': '2023-11-24T06:24:18.843946+0000', 'last_fullsized': '2023-11-24T06:24:18.843946+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.854564+0000', 'last_change': '2023-11-24T06:24:18.854564+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.854564+0000', 'last_undegraded': '2023-11-24T06:24:18.854564+0000', 'last_fullsized': '2023-11-24T06:24:18.854564+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-11-24T06:24:24.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850064+0000', 'last_change': '2023-11-24T06:24:18.850064+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850064+0000', 'last_undegraded': '2023-11-24T06:24:18.850064+0000', 'last_fullsized': '2023-11-24T06:24:18.850064+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850695+0000', 'last_change': '2023-11-24T06:24:18.850695+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850695+0000', 'last_undegraded': '2023-11-24T06:24:18.850695+0000', 'last_fullsized': '2023-11-24T06:24:18.850695+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.845998+0000', 'last_change': '2023-11-24T06:24:18.845998+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.845998+0000', 'last_undegraded': '2023-11-24T06:24:18.845998+0000', 'last_fullsized': '2023-11-24T06:24:18.845998+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849115+0000', 'last_change': '2023-11-24T06:24:18.849115+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849115+0000', 'last_undegraded': '2023-11-24T06:24:18.849115+0000', 'last_fullsized': '2023-11-24T06:24:18.849115+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851388+0000', 'last_change': '2023-11-24T06:24:18.851388+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851388+0000', 'last_undegraded': '2023-11-24T06:24:18.851388+0000', 'last_fullsized': '2023-11-24T06:24:18.851388+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-11-24T06:24:24.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851520+0000', 'last_change': '2023-11-24T06:24:18.851520+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851520+0000', 'last_undegraded': '2023-11-24T06:24:18.851520+0000', 'last_fullsized': '2023-11-24T06:24:18.851520+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849982+0000', 'last_change': '2023-11-24T06:24:18.849982+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849982+0000', 'last_undegraded': '2023-11-24T06:24:18.849982+0000', 'last_fullsized': '2023-11-24T06:24:18.849982+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.847930+0000', 'last_change': '2023-11-24T06:24:18.847930+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.847930+0000', 'last_undegraded': '2023-11-24T06:24:18.847930+0000', 'last_fullsized': '2023-11-24T06:24:18.847930+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.850083+0000', 'last_change': '2023-11-24T06:24:18.850083+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.850083+0000', 'last_undegraded': '2023-11-24T06:24:18.850083+0000', 'last_fullsized': '2023-11-24T06:24:18.850083+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.851624+0000', 'last_change': '2023-11-24T06:24:18.851624+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.851624+0000', 'last_undegraded': '2023-11-24T06:24:18.851624+0000', 'last_fullsized': '2023-11-24T06:24:18.851624+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:24:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-11-24T06:24:24.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.846419+0000', 'last_change': '2023-11-24T06:24:18.846419+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.846419+0000', 'last_undegraded': '2023-11-24T06:24:18.846419+0000', 'last_fullsized': '2023-11-24T06:24:18.846419+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:24:24.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-11-24T06:24:24.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:24:18.849459+0000', 'last_change': '2023-11-24T06:24:18.849459+0000', 'last_active': '2023-11-24T06:24:18.834768+0000', 'last_peered': '2023-11-24T06:24:18.834768+0000', 'last_clean': '2023-11-24T06:24:18.834768+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:24:18.849459+0000', 'last_undegraded': '2023-11-24T06:24:18.849459+0000', 'last_fullsized': '2023-11-24T06:24:18.849459+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'last_clean_scrub_stamp': '2023-11-24T06:24:18.834768+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:24:24.740 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T06:24:24.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.879+0000 7fd93e05b700 1 Processor -- start 2023-11-24T06:24:24.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.879+0000 7fd93e05b700 1 -- start start 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd93e05b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd9380d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd93e05b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd938162950 con 0x7fd9380b2fc0 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd93e05b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380d2d10 con 0x7fd9380d44e0 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd93e05b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380cefd0 con 0x7fd93815dfc0 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd936ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd9380b2fc0 0x7fd938161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53540/0 (socket says 172.21.15.114:53540) 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd936ffd700 1 -- 172.21.15.114:0/2389988046 learned_addr learned my addr 172.21.15.114:0/2389988046 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.880+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd9380d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837324377 0 0) 0x7fd938162950 con 0x7fd9380b2fc0 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9240035b0 con 0x7fd9380b2fc0 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416128044 0 0) 0x7fd9240035b0 con 0x7fd9380b2fc0 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 msgr2=0x7fd9380d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd9380d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 >> v1:172.21.15.172:6789/0 conn(0x7fd9380d44e0 legacy=0x7fd938162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380cf2b0 con 0x7fd9380b2fc0 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fd920001670 con 0x7fd9380b2fc0 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd9380d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 >> v1:172.21.15.114:6789/0 conn(0x7fd9380b2fc0 legacy=0x7fd938161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924003a30 0x7fd924003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd924004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd924004d90 0x7fd924005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380cefd0 con 0x7fd924004d90 2023-11-24T06:24:24.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92400baf0 con 0x7fd924003a30 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92400bcb0 con 0x7fd93815dfc0 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd924004d90 0x7fd924005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924003a30 0x7fd924003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd936ffd700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd924004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 -- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 msgr2=0x7fd924004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd924004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 -- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924003a30 msgr2=0x7fd924003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:24.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924003a30 0x7fd924003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 -- 172.21.15.114:0/2389988046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280094a0 con 0x7fd924004d90 2023-11-24T06:24:24.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd936ffd700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd93815dfc0 0x7fd924004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:24.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924003a30 0x7fd924003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:24.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.881+0000 7fd937fff700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd924004d90 0x7fd924005210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c160 tx=0x7fd924005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2727c637abee5e01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.883+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930001db0 con 0x7fd924004d90 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.883+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93000cca0 con 0x7fd924004d90 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.883+0000 7fd9367fc700 1 -- 172.21.15.114:0/2389988046 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930003c40 con 0x7fd924004d90 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 -- 172.21.15.114:0/2389988046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd924004d90 msgr2=0x7fd924005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2389988046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd924004d90 0x7fd924005210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c160 tx=0x7fd924005880 comp rx=0 tx=0).stop 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd937fff700 1 -- 172.21.15.114:0/2389988046 reap_dead start 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 -- 172.21.15.114:0/2389988046 shutdown_connections 2023-11-24T06:24:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 -- 172.21.15.114:0/2389988046 >> 172.21.15.114:0/2389988046 conn(0x7fd93806b410 msgr2=0x7fd93806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 -- 172.21.15.114:0/2389988046 shutdown_connections 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.884+0000 7fd93e05b700 1 -- 172.21.15.114:0/2389988046 wait complete. 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.885+0000 7fd93e05b700 1 Processor -- start 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.886+0000 7fd93e05b700 1 -- start start 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd938160630 0x7fd9380c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9380c94e0 0x7fd93815e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ca6c0 con 0x7fd9380c9960 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380c7360 con 0x7fd9380c94e0 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd938160a10 con 0x7fd938160630 2023-11-24T06:24:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37844/0 (socket says 172.21.15.114:37844) 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 -- 172.21.15.114:0/2663192759 learned_addr learned my addr 172.21.15.114:0/2663192759 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd936ffd700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9380c94e0 0x7fd93815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd938160630 0x7fd9380c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 -- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd938160630 msgr2=0x7fd9380c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd938160630 0x7fd9380c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 -- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9380c94e0 msgr2=0x7fd93815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9380c94e0 0x7fd93815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 -- 172.21.15.114:0/2663192759 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380c9e20 con 0x7fd9380c9960 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd937fff700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd92400bac0 tx=0x7fd9300014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930003c10 con 0x7fd9380c9960 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930003d70 con 0x7fd9380c9960 2023-11-24T06:24:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9380ca100 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930005310 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.887+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd938153980 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.888+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9380b3620 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.889+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd930019040 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.889+0000 7fd92ffff700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 0x7fd91806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.889+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd93008a1e0 con 0x7fd9380c9960 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.889+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 0x7fd91806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.889+0000 7fd9377fe700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 0x7fd91806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd928009470 tx=0x7fd928008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:24.894+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd930055440 con 0x7fd9380c9960 2023-11-24T06:24:25.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.160+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd938067f60 con 0x7fd9380c9960 2023-11-24T06:24:25.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.160+0000 7fd92ffff700 1 -- 172.21.15.114:0/2663192759 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1031 (secure 0 0 0) 0x7fd930058a60 con 0x7fd9380c9960 2023-11-24T06:24:25.164 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:25.164 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":33,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1700807043,"num_in_osds":8,"osd_in_since":1700807034,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224075776,"bytes_avail":772870037504,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4263}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-24T06:23:52.086486+0000","services":{}},"progress_events":{}} 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 msgr2=0x7fd91806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 0x7fd91806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd928009470 tx=0x7fd928008040 comp rx=0 tx=0).stop 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 msgr2=0x7fd93815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd92400bac0 tx=0x7fd9300014a0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 shutdown_connections 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd91806aed0 0x7fd91806d390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9380c9960 0x7fd93815ee40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd938160630 0x7fd9380c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 --2- 172.21.15.114:0/2663192759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9380c94e0 0x7fd93815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 >> 172.21.15.114:0/2663192759 conn(0x7fd93806b410 msgr2=0x7fd93815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:25.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 shutdown_connections 2023-11-24T06:24:25.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.162+0000 7fd93e05b700 1 -- 172.21.15.114:0/2663192759 wait complete. 2023-11-24T06:24:25.177 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T06:24:25.178 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:25.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.295+0000 7f7694134700 1 Processor -- start 2023-11-24T06:24:25.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.295+0000 7f7694134700 1 -- start start 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f768c13df40 0x7f768c140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 0x7f768c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c140f10 con 0x7f768c13df40 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c0d0950 con 0x7f768c140ad0 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f7694134700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0cf370 con 0x7f768c13db60 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.296+0000 7f76916cf700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f768c13df40 0x7f768c140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f768c13df40 0x7f768c140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37852/0 (socket says 172.21.15.114:37852) 2023-11-24T06:24:25.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 -- 172.21.15.114:0/955482517 learned_addr learned my addr 172.21.15.114:0/955482517 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f7691ed0700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76926d1700 1 --2- 172.21.15.114:0/955482517 >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 0x7f768c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 -- 172.21.15.114:0/955482517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 msgr2=0x7f768c0d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c0d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 -- 172.21.15.114:0/955482517 >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 msgr2=0x7f768c15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 --2- 172.21.15.114:0/955482517 >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 0x7f768c15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 -- 172.21.15.114:0/955482517 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0cf650 con 0x7f768c13df40 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76926d1700 1 --2- 172.21.15.114:0/955482517 >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 0x7f768c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f7691ed0700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c0d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f76916cf700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c13df40 0x7f768c140590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f768c0714c0 tx=0x7f767c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf368fa57184e20 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f7690ece700 1 -- 172.21.15.114:0/955482517 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00b870 con 0x7f768c13df40 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f7690ece700 1 -- 172.21.15.114:0/955482517 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c00b9d0 con 0x7f768c13df40 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.297+0000 7f7690ece700 1 -- 172.21.15.114:0/955482517 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00bd60 con 0x7f768c13df40 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 -- 172.21.15.114:0/955482517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c13df40 msgr2=0x7f768c140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c13df40 0x7f768c140590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f768c0714c0 tx=0x7f767c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 -- 172.21.15.114:0/955482517 shutdown_connections 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c0d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 --2- 172.21.15.114:0/955482517 >> v2:172.21.15.172:3300/0 conn(0x7f768c140ad0 0x7f768c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 --2- 172.21.15.114:0/955482517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c13df40 0x7f768c140590 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 -- 172.21.15.114:0/955482517 >> 172.21.15.114:0/955482517 conn(0x7f768c072a20 msgr2=0x7f768c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:25.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 -- 172.21.15.114:0/955482517 shutdown_connections 2023-11-24T06:24:25.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.298+0000 7f7694134700 1 -- 172.21.15.114:0/955482517 wait complete. 2023-11-24T06:24:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.299+0000 7f7694134700 1 Processor -- start 2023-11-24T06:24:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.299+0000 7f7694134700 1 -- start start 2023-11-24T06:24:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c13df40 0x7f768c1eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c13e360 con 0x7f768c140ad0 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c153c50 con 0x7f768c13df40 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7694134700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c153f30 con 0x7f768c13db60 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37858/0 (socket says 172.21.15.114:37858) 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 -- 172.21.15.114:0/1284830174 learned_addr learned my addr 172.21.15.114:0/1284830174 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76916cf700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c13df40 0x7f768c1eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f7691ed0700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 -- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 msgr2=0x7f768c138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c138d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 -- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c13df40 msgr2=0x7f768c1eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c13df40 0x7f768c1eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.300+0000 7f76926d1700 1 -- 172.21.15.114:0/1284830174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c009550 con 0x7f768c140ad0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f76926d1700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f768400cd50 tx=0x7f768400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400f480 con 0x7f768c140ad0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768400f600 con 0x7f768c140ad0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400f9b0 con 0x7f768c140ad0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c13e5b0 con 0x7f768c140ad0 2023-11-24T06:24:25.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.301+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c0b58e0 con 0x7f768c140ad0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.302+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c13ead0 con 0x7f768c140ad0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.302+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f768400b040 con 0x7f768c140ad0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.302+0000 7f76827fc700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 0x7f767006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.303+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7684089d80 con 0x7f768c140ad0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.303+0000 7f7691ed0700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 0x7f767006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.303+0000 7f7691ed0700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 0x7f767006d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7688006fd0 tx=0x7f7688008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.307+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f768c13ead0 con 0x7f768c140ad0 2023-11-24T06:24:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.493+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f768c1540a0 con 0x7f767006aed0 2023-11-24T06:24:25.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.500+0000 7f76827fc700 1 -- 172.21.15.114:0/1284830174 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259383 (secure 0 0 0) 0x7f768c1540a0 con 0x7f767006aed0 2023-11-24T06:24:25.506 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:25.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 msgr2=0x7f767006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 0x7f767006d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7688006fd0 tx=0x7f7688008040 comp rx=0 tx=0).stop 2023-11-24T06:24:25.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 msgr2=0x7f768c1ef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f768400cd50 tx=0x7f768400a560 comp rx=0 tx=0).stop 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 shutdown_connections 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f767006aed0 0x7f767006d390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f768c140ad0 0x7f768c1ef340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f768c13db60 0x7f768c138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 --2- 172.21.15.114:0/1284830174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f768c13df40 0x7f768c1eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.502+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 >> 172.21.15.114:0/1284830174 conn(0x7f768c072a20 msgr2=0x7f768c0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.503+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 shutdown_connections 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:25.503+0000 7f7694134700 1 -- 172.21.15.114:0/1284830174 wait complete. 2023-11-24T06:24:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:25.518 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:24:23.708302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218824,"kb_used_data":2640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755896,"statfs":{"total":773094113280,"available":772870037504,"internally_reserved":0,"allocated":2703360,"data_stored":1668234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845901+0000","last_change":"2023-11-24T06:24:18.845901+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845901+0000","last_undegraded":"2023-11-24T06:24:18.845901+0000","last_fullsized":"2023-11-24T06:24:18.845901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851607+0000","last_change":"2023-11-24T06:24:18.851607+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851607+0000","last_undegraded":"2023-11-24T06:24:18.851607+0000","last_fullsized":"2023-11-24T06:24:18.851607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846763+0000","last_change":"2023-11-24T06:24:18.846763+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846763+0000","last_undegraded":"2023-11-24T06:24:18.846763+0000","last_fullsized":"2023-11-24T06:24:18.846763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850784+0000","last_change":"2023-11-24T06:24:18.850784+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850784+0000","last_undegraded":"2023-11-24T06:24:18.850784+0000","last_fullsized":"2023-11-24T06:24:18.850784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844740+0000","last_change":"2023-11-24T06:24:18.844740+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844740+0000","last_undegraded":"2023-11-24T06:24:18.844740+0000","last_fullsized":"2023-11-24T06:24:18.844740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.853282+0000","last_change":"2023-11-24T06:24:18.853282+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.853282+0000","last_undegraded":"2023-11-24T06:24:18.853282+0000","last_fullsized":"2023-11-24T06:24:18.853282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845753+0000","last_change":"2023-11-24T06:24:18.845753+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845753+0000","last_undegraded":"2023-11-24T06:24:18.845753+0000","last_fullsized":"2023-11-24T06:24:18.845753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851451+0000","last_change":"2023-11-24T06:24:18.851451+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851451+0000","last_undegraded":"2023-11-24T06:24:18.851451+0000","last_fullsized":"2023-11-24T06:24:18.851451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845084+0000","last_change":"2023-11-24T06:24:18.845084+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845084+0000","last_undegraded":"2023-11-24T06:24:18.845084+0000","last_fullsized":"2023-11-24T06:24:18.845084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849004+0000","last_change":"2023-11-24T06:24:18.849004+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849004+0000","last_undegraded":"2023-11-24T06:24:18.849004+0000","last_fullsized":"2023-11-24T06:24:18.849004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850853+0000","last_change":"2023-11-24T06:24:18.850853+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850853+0000","last_undegraded":"2023-11-24T06:24:18.850853+0000","last_fullsized":"2023-11-24T06:24:18.850853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844967+0000","last_change":"2023-11-24T06:24:18.844967+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844967+0000","last_undegraded":"2023-11-24T06:24:18.844967+0000","last_fullsized":"2023-11-24T06:24:18.844967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851767+0000","last_change":"2023-11-24T06:24:18.851767+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851767+0000","last_undegraded":"2023-11-24T06:24:18.851767+0000","last_fullsized":"2023-11-24T06:24:18.851767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846586+0000","last_change":"2023-11-24T06:24:18.846586+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846586+0000","last_undegraded":"2023-11-24T06:24:18.846586+0000","last_fullsized":"2023-11-24T06:24:18.846586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851597+0000","last_change":"2023-11-24T06:24:18.851597+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851597+0000","last_undegraded":"2023-11-24T06:24:18.851597+0000","last_fullsized":"2023-11-24T06:24:18.851597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851790+0000","last_change":"2023-11-24T06:24:18.851790+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851790+0000","last_undegraded":"2023-11-24T06:24:18.851790+0000","last_fullsized":"2023-11-24T06:24:18.851790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848814+0000","last_change":"2023-11-24T06:24:18.848814+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848814+0000","last_undegraded":"2023-11-24T06:24:18.848814+0000","last_fullsized":"2023-11-24T06:24:18.848814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850442+0000","last_change":"2023-11-24T06:24:18.850442+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850442+0000","last_undegraded":"2023-11-24T06:24:18.850442+0000","last_fullsized":"2023-11-24T06:24:18.850442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846097+0000","last_change":"2023-11-24T06:24:18.846097+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846097+0000","last_undegraded":"2023-11-24T06:24:18.846097+0000","last_fullsized":"2023-11-24T06:24:18.846097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859398+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:18.859398+0000","last_peered":"2023-11-24T06:24:18.859398+0000","last_clean":"2023-11-24T06:24:18.859398+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:18.859398+0000","last_undegraded":"2023-11-24T06:24:18.859398+0000","last_fullsized":"2023-11-24T06:24:18.859398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849020+0000","last_change":"2023-11-24T06:24:18.849020+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849020+0000","last_undegraded":"2023-11-24T06:24:18.849020+0000","last_fullsized":"2023-11-24T06:24:18.849020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859724+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:18.859724+0000","last_peered":"2023-11-24T06:24:18.859724+0000","last_clean":"2023-11-24T06:24:18.859724+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:18.859724+0000","last_undegraded":"2023-11-24T06:24:18.859724+0000","last_fullsized":"2023-11-24T06:24:18.859724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847546+0000","last_change":"2023-11-24T06:24:18.847546+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847546+0000","last_undegraded":"2023-11-24T06:24:18.847546+0000","last_fullsized":"2023-11-24T06:24:18.847546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862355+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:18.862355+0000","last_peered":"2023-11-24T06:24:18.862355+0000","last_clean":"2023-11-24T06:24:18.862355+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:18.862355+0000","last_undegraded":"2023-11-24T06:24:18.862355+0000","last_fullsized":"2023-11-24T06:24:18.862355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.856218+0000","last_change":"2023-11-24T06:24:18.856218+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.856218+0000","last_undegraded":"2023-11-24T06:24:18.856218+0000","last_fullsized":"2023-11-24T06:24:18.856218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864140+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:18.864140+0000","last_peered":"2023-11-24T06:24:18.864140+0000","last_clean":"2023-11-24T06:24:18.864140+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:18.864140+0000","last_undegraded":"2023-11-24T06:24:18.864140+0000","last_fullsized":"2023-11-24T06:24:18.864140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850116+0000","last_change":"2023-11-24T06:24:18.850116+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850116+0000","last_undegraded":"2023-11-24T06:24:18.850116+0000","last_fullsized":"2023-11-24T06:24:18.850116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865327+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:18.865327+0000","last_peered":"2023-11-24T06:24:18.865327+0000","last_clean":"2023-11-24T06:24:18.865327+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:18.865327+0000","last_undegraded":"2023-11-24T06:24:18.865327+0000","last_fullsized":"2023-11-24T06:24:18.865327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843310+0000","last_change":"2023-11-24T06:24:18.843310+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843310+0000","last_undegraded":"2023-11-24T06:24:18.843310+0000","last_fullsized":"2023-11-24T06:24:18.843310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859116+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:18.859116+0000","last_peered":"2023-11-24T06:24:18.859116+0000","last_clean":"2023-11-24T06:24:18.859116+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:18.859116+0000","last_undegraded":"2023-11-24T06:24:18.859116+0000","last_fullsized":"2023-11-24T06:24:18.859116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845473+0000","last_change":"2023-11-24T06:24:18.845473+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845473+0000","last_undegraded":"2023-11-24T06:24:18.845473+0000","last_fullsized":"2023-11-24T06:24:18.845473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859717+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:18.859717+0000","last_peered":"2023-11-24T06:24:18.859717+0000","last_clean":"2023-11-24T06:24:18.859717+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:18.859717+0000","last_undegraded":"2023-11-24T06:24:18.859717+0000","last_fullsized":"2023-11-24T06:24:18.859717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845987+0000","last_change":"2023-11-24T06:24:18.845987+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845987+0000","last_undegraded":"2023-11-24T06:24:18.845987+0000","last_fullsized":"2023-11-24T06:24:18.845987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863686+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:18.863686+0000","last_peered":"2023-11-24T06:24:18.863686+0000","last_clean":"2023-11-24T06:24:18.863686+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:18.863686+0000","last_undegraded":"2023-11-24T06:24:18.863686+0000","last_fullsized":"2023-11-24T06:24:18.863686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850409+0000","last_change":"2023-11-24T06:24:18.850409+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850409+0000","last_undegraded":"2023-11-24T06:24:18.850409+0000","last_fullsized":"2023-11-24T06:24:18.850409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864305+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:18.864305+0000","last_peered":"2023-11-24T06:24:18.864305+0000","last_clean":"2023-11-24T06:24:18.864305+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:18.864305+0000","last_undegraded":"2023-11-24T06:24:18.864305+0000","last_fullsized":"2023-11-24T06:24:18.864305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850760+0000","last_change":"2023-11-24T06:24:18.850760+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850760+0000","last_undegraded":"2023-11-24T06:24:18.850760+0000","last_fullsized":"2023-11-24T06:24:18.850760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863658+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:18.863658+0000","last_peered":"2023-11-24T06:24:18.863658+0000","last_clean":"2023-11-24T06:24:18.863658+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:18.863658+0000","last_undegraded":"2023-11-24T06:24:18.863658+0000","last_fullsized":"2023-11-24T06:24:18.863658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847096+0000","last_change":"2023-11-24T06:24:18.847096+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847096+0000","last_undegraded":"2023-11-24T06:24:18.847096+0000","last_fullsized":"2023-11-24T06:24:18.847096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.866421+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:18.866421+0000","last_peered":"2023-11-24T06:24:18.866421+0000","last_clean":"2023-11-24T06:24:18.866421+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:18.866421+0000","last_undegraded":"2023-11-24T06:24:18.866421+0000","last_fullsized":"2023-11-24T06:24:18.866421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843113+0000","last_change":"2023-11-24T06:24:18.843113+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843113+0000","last_undegraded":"2023-11-24T06:24:18.843113+0000","last_fullsized":"2023-11-24T06:24:18.843113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863776+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:18.863776+0000","last_peered":"2023-11-24T06:24:18.863776+0000","last_clean":"2023-11-24T06:24:18.863776+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:18.863776+0000","last_undegraded":"2023-11-24T06:24:18.863776+0000","last_fullsized":"2023-11-24T06:24:18.863776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845117+0000","last_change":"2023-11-24T06:24:18.845117+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845117+0000","last_undegraded":"2023-11-24T06:24:18.845117+0000","last_fullsized":"2023-11-24T06:24:18.845117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.856003+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:18.856003+0000","last_peered":"2023-11-24T06:24:18.856003+0000","last_clean":"2023-11-24T06:24:18.856003+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:18.856003+0000","last_undegraded":"2023-11-24T06:24:18.856003+0000","last_fullsized":"2023-11-24T06:24:18.856003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845031+0000","last_change":"2023-11-24T06:24:18.845031+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845031+0000","last_undegraded":"2023-11-24T06:24:18.845031+0000","last_fullsized":"2023-11-24T06:24:18.845031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863773+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:18.863773+0000","last_peered":"2023-11-24T06:24:18.863773+0000","last_clean":"2023-11-24T06:24:18.863773+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:18.863773+0000","last_undegraded":"2023-11-24T06:24:18.863773+0000","last_fullsized":"2023-11-24T06:24:18.863773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843918+0000","last_change":"2023-11-24T06:24:18.843918+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843918+0000","last_undegraded":"2023-11-24T06:24:18.843918+0000","last_fullsized":"2023-11-24T06:24:18.843918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.859552+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:18.859552+0000","last_peered":"2023-11-24T06:24:18.859552+0000","last_clean":"2023-11-24T06:24:18.859552+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:18.859552+0000","last_undegraded":"2023-11-24T06:24:18.859552+0000","last_fullsized":"2023-11-24T06:24:18.859552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849274+0000","last_change":"2023-11-24T06:24:18.849274+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849274+0000","last_undegraded":"2023-11-24T06:24:18.849274+0000","last_fullsized":"2023-11-24T06:24:18.849274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865485+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:18.865485+0000","last_peered":"2023-11-24T06:24:18.865485+0000","last_clean":"2023-11-24T06:24:18.865485+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:18.865485+0000","last_undegraded":"2023-11-24T06:24:18.865485+0000","last_fullsized":"2023-11-24T06:24:18.865485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844048+0000","last_change":"2023-11-24T06:24:18.844048+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844048+0000","last_undegraded":"2023-11-24T06:24:18.844048+0000","last_fullsized":"2023-11-24T06:24:18.844048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.858819+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:18.858819+0000","last_peered":"2023-11-24T06:24:18.858819+0000","last_clean":"2023-11-24T06:24:18.858819+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:18.858819+0000","last_undegraded":"2023-11-24T06:24:18.858819+0000","last_fullsized":"2023-11-24T06:24:18.858819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844498+0000","last_change":"2023-11-24T06:24:18.844498+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844498+0000","last_undegraded":"2023-11-24T06:24:18.844498+0000","last_fullsized":"2023-11-24T06:24:18.844498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864328+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:18.864328+0000","last_peered":"2023-11-24T06:24:18.864328+0000","last_clean":"2023-11-24T06:24:18.864328+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:18.864328+0000","last_undegraded":"2023-11-24T06:24:18.864328+0000","last_fullsized":"2023-11-24T06:24:18.864328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851522+0000","last_change":"2023-11-24T06:24:18.851522+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851522+0000","last_undegraded":"2023-11-24T06:24:18.851522+0000","last_fullsized":"2023-11-24T06:24:18.851522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864303+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:18.864303+0000","last_peered":"2023-11-24T06:24:18.864303+0000","last_clean":"2023-11-24T06:24:18.864303+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:18.864303+0000","last_undegraded":"2023-11-24T06:24:18.864303+0000","last_fullsized":"2023-11-24T06:24:18.864303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843387+0000","last_change":"2023-11-24T06:24:18.843387+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843387+0000","last_undegraded":"2023-11-24T06:24:18.843387+0000","last_fullsized":"2023-11-24T06:24:18.843387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.848580+0000","last_change":"2023-11-24T06:24:18.848580+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.848580+0000","last_undegraded":"2023-11-24T06:24:18.848580+0000","last_fullsized":"2023-11-24T06:24:18.848580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864891+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:18.864891+0000","last_peered":"2023-11-24T06:24:18.864891+0000","last_clean":"2023-11-24T06:24:18.864891+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:18.864891+0000","last_undegraded":"2023-11-24T06:24:18.864891+0000","last_fullsized":"2023-11-24T06:24:18.864891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845534+0000","last_change":"2023-11-24T06:24:18.845534+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845534+0000","last_undegraded":"2023-11-24T06:24:18.845534+0000","last_fullsized":"2023-11-24T06:24:18.845534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864162+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:18.864162+0000","last_peered":"2023-11-24T06:24:18.864162+0000","last_clean":"2023-11-24T06:24:18.864162+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:18.864162+0000","last_undegraded":"2023-11-24T06:24:18.864162+0000","last_fullsized":"2023-11-24T06:24:18.864162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:00.325159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:24:17.839121+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:17.839121+0000","last_peered":"2023-11-24T06:24:17.839121+0000","last_clean":"2023-11-24T06:24:17.839121+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:17.839121+0000","last_undegraded":"2023-11-24T06:24:17.839121+0000","last_fullsized":"2023-11-24T06:24:17.839121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844436+0000","last_change":"2023-11-24T06:24:18.844436+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844436+0000","last_undegraded":"2023-11-24T06:24:18.844436+0000","last_fullsized":"2023-11-24T06:24:18.844436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861746+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:18.861746+0000","last_peered":"2023-11-24T06:24:18.861746+0000","last_clean":"2023-11-24T06:24:18.861746+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:18.861746+0000","last_undegraded":"2023-11-24T06:24:18.861746+0000","last_fullsized":"2023-11-24T06:24:18.861746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862011+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:18.862011+0000","last_peered":"2023-11-24T06:24:18.862011+0000","last_clean":"2023-11-24T06:24:18.862011+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:18.862011+0000","last_undegraded":"2023-11-24T06:24:18.862011+0000","last_fullsized":"2023-11-24T06:24:18.862011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850677+0000","last_change":"2023-11-24T06:24:18.850677+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850677+0000","last_undegraded":"2023-11-24T06:24:18.850677+0000","last_fullsized":"2023-11-24T06:24:18.850677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861727+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:18.861727+0000","last_peered":"2023-11-24T06:24:18.861727+0000","last_clean":"2023-11-24T06:24:18.861727+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:18.861727+0000","last_undegraded":"2023-11-24T06:24:18.861727+0000","last_fullsized":"2023-11-24T06:24:18.861727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:31:37.780677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849533+0000","last_change":"2023-11-24T06:24:18.849533+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849533+0000","last_undegraded":"2023-11-24T06:24:18.849533+0000","last_fullsized":"2023-11-24T06:24:18.849533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864318+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:18.864318+0000","last_peered":"2023-11-24T06:24:18.864318+0000","last_clean":"2023-11-24T06:24:18.864318+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:18.864318+0000","last_undegraded":"2023-11-24T06:24:18.864318+0000","last_fullsized":"2023-11-24T06:24:18.864318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844472+0000","last_change":"2023-11-24T06:24:18.844472+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844472+0000","last_undegraded":"2023-11-24T06:24:18.844472+0000","last_fullsized":"2023-11-24T06:24:18.844472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862045+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:18.862045+0000","last_peered":"2023-11-24T06:24:18.862045+0000","last_clean":"2023-11-24T06:24:18.862045+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:18.862045+0000","last_undegraded":"2023-11-24T06:24:18.862045+0000","last_fullsized":"2023-11-24T06:24:18.862045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843078+0000","last_change":"2023-11-24T06:24:18.843078+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843078+0000","last_undegraded":"2023-11-24T06:24:18.843078+0000","last_fullsized":"2023-11-24T06:24:18.843078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.863798+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:18.863798+0000","last_peered":"2023-11-24T06:24:18.863798+0000","last_clean":"2023-11-24T06:24:18.863798+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:18.863798+0000","last_undegraded":"2023-11-24T06:24:18.863798+0000","last_fullsized":"2023-11-24T06:24:18.863798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.850868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845239+0000","last_change":"2023-11-24T06:24:18.845239+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845239+0000","last_undegraded":"2023-11-24T06:24:18.845239+0000","last_fullsized":"2023-11-24T06:24:18.845239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.864361+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:18.864361+0000","last_peered":"2023-11-24T06:24:18.864361+0000","last_clean":"2023-11-24T06:24:18.864361+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:18.864361+0000","last_undegraded":"2023-11-24T06:24:18.864361+0000","last_fullsized":"2023-11-24T06:24:18.864361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845746+0000","last_change":"2023-11-24T06:24:18.845746+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845746+0000","last_undegraded":"2023-11-24T06:24:18.845746+0000","last_fullsized":"2023-11-24T06:24:18.845746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.861865+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:18.861865+0000","last_peered":"2023-11-24T06:24:18.861865+0000","last_clean":"2023-11-24T06:24:18.861865+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:18.861865+0000","last_undegraded":"2023-11-24T06:24:18.861865+0000","last_fullsized":"2023-11-24T06:24:18.861865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850115+0000","last_change":"2023-11-24T06:24:18.850115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850115+0000","last_undegraded":"2023-11-24T06:24:18.850115+0000","last_fullsized":"2023-11-24T06:24:18.850115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865516+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:18.865516+0000","last_peered":"2023-11-24T06:24:18.865516+0000","last_clean":"2023-11-24T06:24:18.865516+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:18.865516+0000","last_undegraded":"2023-11-24T06:24:18.865516+0000","last_fullsized":"2023-11-24T06:24:18.865516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:13.401194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.844391+0000","last_change":"2023-11-24T06:24:18.844391+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.844391+0000","last_undegraded":"2023-11-24T06:24:18.844391+0000","last_fullsized":"2023-11-24T06:24:18.844391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.865112+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:18.865112+0000","last_peered":"2023-11-24T06:24:18.865112+0000","last_clean":"2023-11-24T06:24:18.865112+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:18.865112+0000","last_undegraded":"2023-11-24T06:24:18.865112+0000","last_fullsized":"2023-11-24T06:24:18.865112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854299+0000","last_change":"2023-11-24T06:24:18.854299+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854299+0000","last_undegraded":"2023-11-24T06:24:18.854299+0000","last_fullsized":"2023-11-24T06:24:18.854299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:24:18.862641+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:18.862641+0000","last_peered":"2023-11-24T06:24:18.862641+0000","last_clean":"2023-11-24T06:24:18.862641+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:18.862641+0000","last_undegraded":"2023-11-24T06:24:18.862641+0000","last_fullsized":"2023-11-24T06:24:18.862641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.843946+0000","last_change":"2023-11-24T06:24:18.843946+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.843946+0000","last_undegraded":"2023-11-24T06:24:18.843946+0000","last_fullsized":"2023-11-24T06:24:18.843946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.854564+0000","last_change":"2023-11-24T06:24:18.854564+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.854564+0000","last_undegraded":"2023-11-24T06:24:18.854564+0000","last_fullsized":"2023-11-24T06:24:18.854564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850064+0000","last_change":"2023-11-24T06:24:18.850064+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850064+0000","last_undegraded":"2023-11-24T06:24:18.850064+0000","last_fullsized":"2023-11-24T06:24:18.850064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850695+0000","last_change":"2023-11-24T06:24:18.850695+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850695+0000","last_undegraded":"2023-11-24T06:24:18.850695+0000","last_fullsized":"2023-11-24T06:24:18.850695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.845998+0000","last_change":"2023-11-24T06:24:18.845998+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.845998+0000","last_undegraded":"2023-11-24T06:24:18.845998+0000","last_fullsized":"2023-11-24T06:24:18.845998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849115+0000","last_change":"2023-11-24T06:24:18.849115+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849115+0000","last_undegraded":"2023-11-24T06:24:18.849115+0000","last_fullsized":"2023-11-24T06:24:18.849115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851388+0000","last_change":"2023-11-24T06:24:18.851388+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851388+0000","last_undegraded":"2023-11-24T06:24:18.851388+0000","last_fullsized":"2023-11-24T06:24:18.851388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851520+0000","last_change":"2023-11-24T06:24:18.851520+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851520+0000","last_undegraded":"2023-11-24T06:24:18.851520+0000","last_fullsized":"2023-11-24T06:24:18.851520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849982+0000","last_change":"2023-11-24T06:24:18.849982+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849982+0000","last_undegraded":"2023-11-24T06:24:18.849982+0000","last_fullsized":"2023-11-24T06:24:18.849982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.847930+0000","last_change":"2023-11-24T06:24:18.847930+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.847930+0000","last_undegraded":"2023-11-24T06:24:18.847930+0000","last_fullsized":"2023-11-24T06:24:18.847930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.850083+0000","last_change":"2023-11-24T06:24:18.850083+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.850083+0000","last_undegraded":"2023-11-24T06:24:18.850083+0000","last_fullsized":"2023-11-24T06:24:18.850083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.851624+0000","last_change":"2023-11-24T06:24:18.851624+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.851624+0000","last_undegraded":"2023-11-24T06:24:18.851624+0000","last_fullsized":"2023-11-24T06:24:18.851624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.846419+0000","last_change":"2023-11-24T06:24:18.846419+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.846419+0000","last_undegraded":"2023-11-24T06:24:18.846419+0000","last_fullsized":"2023-11-24T06:24:18.846419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-11-24T06:24:18.849459+0000","last_change":"2023-11-24T06:24:18.849459+0000","last_active":"2023-11-24T06:24:18.834768+0000","last_peered":"2023-11-24T06:24:18.834768+0000","last_clean":"2023-11-24T06:24:18.834768+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:24:18.849459+0000","last_undegraded":"2023-11-24T06:24:18.849459+0000","last_fullsized":"2023-11-24T06:24:18.849459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":646599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":652874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":62506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:28.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.654+0000 7faa9aa16700 1 Processor -- start 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.654+0000 7faa9aa16700 1 -- start start 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa9aa16700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faa9406c710 0x7faa9416d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa9aa16700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa940b46e0 con 0x7faa94162240 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa9aa16700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa940b49c0 con 0x7faa940ae4e0 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa9aa16700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa94166f50 con 0x7faa9406c710 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa98fb3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faa9406c710 0x7faa9416d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:28.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa93fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7faa94162240 0x7faa940afed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53570/0 (socket says 172.21.15.114:53570) 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa93fff700 1 -- 172.21.15.114:0/1669607932 learned_addr learned my addr 172.21.15.114:0/1669607932 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa98fb3700 1 -- 172.21.15.114:0/1669607932 >> v1:172.21.15.172:6789/0 conn(0x7faa940ae4e0 legacy=0x7faa9406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa98fb3700 1 -- 172.21.15.114:0/1669607932 >> v1:172.21.15.114:6789/0 conn(0x7faa94162240 legacy=0x7faa940afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa98fb3700 1 -- 172.21.15.114:0/1669607932 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940b4ca0 con 0x7faa9406c710 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa92ffd700 1 -- 172.21.15.114:0/1669607932 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585043210 0 0) 0x7faa940b46e0 con 0x7faa94162240 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.655+0000 7faa98fb3700 1 --2- 172.21.15.114:0/1669607932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa9406c710 0x7faa9416d2d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009d90 tx=0x7faa8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ecb51fc51d7754 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa92ffd700 1 -- 172.21.15.114:0/1669607932 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00e070 con 0x7faa9406c710 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa92ffd700 1 -- 172.21.15.114:0/1669607932 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00bb90 con 0x7faa9406c710 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa92ffd700 1 -- 172.21.15.114:0/1669607932 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c003a20 con 0x7faa9406c710 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 -- 172.21.15.114:0/1669607932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa9406c710 msgr2=0x7faa9416d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 --2- 172.21.15.114:0/1669607932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa9406c710 0x7faa9416d2d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009d90 tx=0x7faa8c00b330 comp rx=0 tx=0).stop 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 -- 172.21.15.114:0/1669607932 shutdown_connections 2023-11-24T06:24:28.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 --2- 172.21.15.114:0/1669607932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa9406c710 0x7faa9416d2d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 -- 172.21.15.114:0/1669607932 >> 172.21.15.114:0/1669607932 conn(0x7faa94071b90 msgr2=0x7faa94071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 -- 172.21.15.114:0/1669607932 shutdown_connections 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.656+0000 7faa9aa16700 1 -- 172.21.15.114:0/1669607932 wait complete. 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.658+0000 7faa9aa16700 1 Processor -- start 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.658+0000 7faa9aa16700 1 -- start start 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 0x7faa94162980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 0x7faa94160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94163400 con 0x7faa940ae4e0 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94153450 con 0x7faa94162240 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa941630b0 con 0x7faa9406c710 2023-11-24T06:24:28.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37870/0 (socket says 172.21.15.114:37870) 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 -- 172.21.15.114:0/3956072699 learned_addr learned my addr 172.21.15.114:0/3956072699 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa98fb3700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 0x7faa94160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa93fff700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 0x7faa94162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 -- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 msgr2=0x7faa94162980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 0x7faa94162980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 -- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 msgr2=0x7faa94160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 0x7faa94160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 -- 172.21.15.114:0/3956072699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c009a70 con 0x7faa940ae4e0 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa98fb3700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 0x7faa94160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa93fff700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 0x7faa94162980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa937fe700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c550 tx=0x7faa7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:28.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c016070 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c006860 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa940cfe00 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.659+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c006bf0 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.660+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa940d03c0 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.660+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa940753f0 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.661+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa7c006d50 con 0x7faa940ae4e0 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.661+0000 7faa90ff9700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 0x7faa7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:28.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.661+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa7c088ca0 con 0x7faa940ae4e0 2023-11-24T06:24:28.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.661+0000 7faa93fff700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 0x7faa7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:28.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.662+0000 7faa93fff700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 0x7faa7406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa84000f60 tx=0x7faa84007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:28.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.666+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa7c053de0 con 0x7faa940ae4e0 2023-11-24T06:24:28.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.856+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa940b5260 con 0x7faa7406aed0 2023-11-24T06:24:28.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.863+0000 7faa90ff9700 1 -- 172.21.15.114:0/3956072699 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264702 (secure 0 0 0) 0x7faa84005cd0 con 0x7faa7406aed0 2023-11-24T06:24:28.869 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:28.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 msgr2=0x7faa7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 0x7faa7406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa84000f60 tx=0x7faa84007e50 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 msgr2=0x7faa94165200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c550 tx=0x7faa7c00c910 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 shutdown_connections 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faa7406aed0 0x7faa7406d390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa94162240 0x7faa94160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa9406c710 0x7faa94162980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 --2- 172.21.15.114:0/3956072699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa940ae4e0 0x7faa94165200 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 >> 172.21.15.114:0/3956072699 conn(0x7faa94071b90 msgr2=0x7faa9406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 shutdown_connections 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:28.866+0000 7faa9aa16700 1 -- 172.21.15.114:0/3956072699 wait complete. 2023-11-24T06:24:28.872 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:28.881 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-24T06:24:27.709421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219108,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755612,"statfs":{"total":773094113280,"available":772869746688,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001963"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854842+0000","last_change":"2023-11-24T06:24:19.869889+0000","last_active":"2023-11-24T06:24:20.854842+0000","last_peered":"2023-11-24T06:24:20.854842+0000","last_clean":"2023-11-24T06:24:20.854842+0000","last_became_active":"2023-11-24T06:24:19.869608+0000","last_became_peered":"2023-11-24T06:24:19.869608+0000","last_unstale":"2023-11-24T06:24:20.854842+0000","last_undegraded":"2023-11-24T06:24:20.854842+0000","last_fullsized":"2023-11-24T06:24:20.854842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:48.551881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853246+0000","last_change":"2023-11-24T06:24:19.872814+0000","last_active":"2023-11-24T06:24:20.853246+0000","last_peered":"2023-11-24T06:24:20.853246+0000","last_clean":"2023-11-24T06:24:20.853246+0000","last_became_active":"2023-11-24T06:24:19.872430+0000","last_became_peered":"2023-11-24T06:24:19.872430+0000","last_unstale":"2023-11-24T06:24:20.853246+0000","last_undegraded":"2023-11-24T06:24:20.853246+0000","last_fullsized":"2023-11-24T06:24:20.853246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979460+0000","last_change":"2023-11-24T06:24:19.869057+0000","last_active":"2023-11-24T06:24:20.979460+0000","last_peered":"2023-11-24T06:24:20.979460+0000","last_clean":"2023-11-24T06:24:20.979460+0000","last_became_active":"2023-11-24T06:24:19.868779+0000","last_became_peered":"2023-11-24T06:24:19.868779+0000","last_unstale":"2023-11-24T06:24:20.979460+0000","last_undegraded":"2023-11-24T06:24:20.979460+0000","last_fullsized":"2023-11-24T06:24:20.979460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054845+0000","last_change":"2023-11-24T06:24:19.875414+0000","last_active":"2023-11-24T06:24:21.054845+0000","last_peered":"2023-11-24T06:24:21.054845+0000","last_clean":"2023-11-24T06:24:21.054845+0000","last_became_active":"2023-11-24T06:24:19.875156+0000","last_became_peered":"2023-11-24T06:24:19.875156+0000","last_unstale":"2023-11-24T06:24:21.054845+0000","last_undegraded":"2023-11-24T06:24:21.054845+0000","last_fullsized":"2023-11-24T06:24:21.054845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854274+0000","last_change":"2023-11-24T06:24:19.862612+0000","last_active":"2023-11-24T06:24:20.854274+0000","last_peered":"2023-11-24T06:24:20.854274+0000","last_clean":"2023-11-24T06:24:20.854274+0000","last_became_active":"2023-11-24T06:24:19.862302+0000","last_became_peered":"2023-11-24T06:24:19.862302+0000","last_unstale":"2023-11-24T06:24:20.854274+0000","last_undegraded":"2023-11-24T06:24:20.854274+0000","last_fullsized":"2023-11-24T06:24:20.854274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054394+0000","last_change":"2023-11-24T06:24:19.871642+0000","last_active":"2023-11-24T06:24:21.054394+0000","last_peered":"2023-11-24T06:24:21.054394+0000","last_clean":"2023-11-24T06:24:21.054394+0000","last_became_active":"2023-11-24T06:24:19.871363+0000","last_became_peered":"2023-11-24T06:24:19.871363+0000","last_unstale":"2023-11-24T06:24:21.054394+0000","last_undegraded":"2023-11-24T06:24:21.054394+0000","last_fullsized":"2023-11-24T06:24:21.054394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854102+0000","last_change":"2023-11-24T06:24:19.865691+0000","last_active":"2023-11-24T06:24:20.854102+0000","last_peered":"2023-11-24T06:24:20.854102+0000","last_clean":"2023-11-24T06:24:20.854102+0000","last_became_active":"2023-11-24T06:24:19.865409+0000","last_became_peered":"2023-11-24T06:24:19.865409+0000","last_unstale":"2023-11-24T06:24:20.854102+0000","last_undegraded":"2023-11-24T06:24:20.854102+0000","last_fullsized":"2023-11-24T06:24:20.854102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904022+0000","last_change":"2023-11-24T06:24:19.866800+0000","last_active":"2023-11-24T06:24:20.904022+0000","last_peered":"2023-11-24T06:24:20.904022+0000","last_clean":"2023-11-24T06:24:20.904022+0000","last_became_active":"2023-11-24T06:24:19.866162+0000","last_became_peered":"2023-11-24T06:24:19.866162+0000","last_unstale":"2023-11-24T06:24:20.904022+0000","last_undegraded":"2023-11-24T06:24:20.904022+0000","last_fullsized":"2023-11-24T06:24:20.904022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854764+0000","last_change":"2023-11-24T06:24:19.867736+0000","last_active":"2023-11-24T06:24:20.854764+0000","last_peered":"2023-11-24T06:24:20.854764+0000","last_clean":"2023-11-24T06:24:20.854764+0000","last_became_active":"2023-11-24T06:24:19.867377+0000","last_became_peered":"2023-11-24T06:24:19.867377+0000","last_unstale":"2023-11-24T06:24:20.854764+0000","last_undegraded":"2023-11-24T06:24:20.854764+0000","last_fullsized":"2023-11-24T06:24:20.854764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553520+0000","last_change":"2023-11-24T06:24:19.870504+0000","last_active":"2023-11-24T06:24:21.553520+0000","last_peered":"2023-11-24T06:24:21.553520+0000","last_clean":"2023-11-24T06:24:21.553520+0000","last_became_active":"2023-11-24T06:24:19.870143+0000","last_became_peered":"2023-11-24T06:24:19.870143+0000","last_unstale":"2023-11-24T06:24:21.553520+0000","last_undegraded":"2023-11-24T06:24:21.553520+0000","last_fullsized":"2023-11-24T06:24:21.553520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553151+0000","last_change":"2023-11-24T06:24:19.873638+0000","last_active":"2023-11-24T06:24:21.553151+0000","last_peered":"2023-11-24T06:24:21.553151+0000","last_clean":"2023-11-24T06:24:21.553151+0000","last_became_active":"2023-11-24T06:24:19.871551+0000","last_became_peered":"2023-11-24T06:24:19.871551+0000","last_unstale":"2023-11-24T06:24:21.553151+0000","last_undegraded":"2023-11-24T06:24:21.553151+0000","last_fullsized":"2023-11-24T06:24:21.553151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855889+0000","last_change":"2023-11-24T06:24:19.867808+0000","last_active":"2023-11-24T06:24:20.855889+0000","last_peered":"2023-11-24T06:24:20.855889+0000","last_clean":"2023-11-24T06:24:20.855889+0000","last_became_active":"2023-11-24T06:24:19.867487+0000","last_became_peered":"2023-11-24T06:24:19.867487+0000","last_unstale":"2023-11-24T06:24:20.855889+0000","last_undegraded":"2023-11-24T06:24:20.855889+0000","last_fullsized":"2023-11-24T06:24:20.855889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904066+0000","last_change":"2023-11-24T06:24:19.870784+0000","last_active":"2023-11-24T06:24:20.904066+0000","last_peered":"2023-11-24T06:24:20.904066+0000","last_clean":"2023-11-24T06:24:20.904066+0000","last_became_active":"2023-11-24T06:24:19.870374+0000","last_became_peered":"2023-11-24T06:24:19.870374+0000","last_unstale":"2023-11-24T06:24:20.904066+0000","last_undegraded":"2023-11-24T06:24:20.904066+0000","last_fullsized":"2023-11-24T06:24:20.904066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979266+0000","last_change":"2023-11-24T06:24:19.865517+0000","last_active":"2023-11-24T06:24:20.979266+0000","last_peered":"2023-11-24T06:24:20.979266+0000","last_clean":"2023-11-24T06:24:20.979266+0000","last_became_active":"2023-11-24T06:24:19.865195+0000","last_became_peered":"2023-11-24T06:24:19.865195+0000","last_unstale":"2023-11-24T06:24:20.979266+0000","last_undegraded":"2023-11-24T06:24:20.979266+0000","last_fullsized":"2023-11-24T06:24:20.979266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852902+0000","last_change":"2023-11-24T06:24:19.867596+0000","last_active":"2023-11-24T06:24:20.852902+0000","last_peered":"2023-11-24T06:24:20.852902+0000","last_clean":"2023-11-24T06:24:20.852902+0000","last_became_active":"2023-11-24T06:24:19.867302+0000","last_became_peered":"2023-11-24T06:24:19.867302+0000","last_unstale":"2023-11-24T06:24:20.852902+0000","last_undegraded":"2023-11-24T06:24:20.852902+0000","last_fullsized":"2023-11-24T06:24:20.852902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054302+0000","last_change":"2023-11-24T06:24:19.871929+0000","last_active":"2023-11-24T06:24:21.054302+0000","last_peered":"2023-11-24T06:24:21.054302+0000","last_clean":"2023-11-24T06:24:21.054302+0000","last_became_active":"2023-11-24T06:24:19.870555+0000","last_became_peered":"2023-11-24T06:24:19.870555+0000","last_unstale":"2023-11-24T06:24:21.054302+0000","last_undegraded":"2023-11-24T06:24:21.054302+0000","last_fullsized":"2023-11-24T06:24:21.054302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054970+0000","last_change":"2023-11-24T06:24:19.871912+0000","last_active":"2023-11-24T06:24:21.054970+0000","last_peered":"2023-11-24T06:24:21.054970+0000","last_clean":"2023-11-24T06:24:21.054970+0000","last_became_active":"2023-11-24T06:24:19.871473+0000","last_became_peered":"2023-11-24T06:24:19.871473+0000","last_unstale":"2023-11-24T06:24:21.054970+0000","last_undegraded":"2023-11-24T06:24:21.054970+0000","last_fullsized":"2023-11-24T06:24:21.054970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553079+0000","last_change":"2023-11-24T06:24:19.873774+0000","last_active":"2023-11-24T06:24:21.553079+0000","last_peered":"2023-11-24T06:24:21.553079+0000","last_clean":"2023-11-24T06:24:21.553079+0000","last_became_active":"2023-11-24T06:24:19.872880+0000","last_became_peered":"2023-11-24T06:24:19.872880+0000","last_unstale":"2023-11-24T06:24:21.553079+0000","last_undegraded":"2023-11-24T06:24:21.553079+0000","last_fullsized":"2023-11-24T06:24:21.553079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854750+0000","last_change":"2023-11-24T06:24:19.863508+0000","last_active":"2023-11-24T06:24:20.854750+0000","last_peered":"2023-11-24T06:24:20.854750+0000","last_clean":"2023-11-24T06:24:20.854750+0000","last_became_active":"2023-11-24T06:24:19.863100+0000","last_became_peered":"2023-11-24T06:24:19.863100+0000","last_unstale":"2023-11-24T06:24:20.854750+0000","last_undegraded":"2023-11-24T06:24:20.854750+0000","last_fullsized":"2023-11-24T06:24:20.854750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.892967+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:19.892967+0000","last_peered":"2023-11-24T06:24:19.892967+0000","last_clean":"2023-11-24T06:24:19.892967+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:19.892967+0000","last_undegraded":"2023-11-24T06:24:19.892967+0000","last_fullsized":"2023-11-24T06:24:19.892967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904290+0000","last_change":"2023-11-24T06:24:19.869582+0000","last_active":"2023-11-24T06:24:20.904290+0000","last_peered":"2023-11-24T06:24:20.904290+0000","last_clean":"2023-11-24T06:24:20.904290+0000","last_became_active":"2023-11-24T06:24:19.869283+0000","last_became_peered":"2023-11-24T06:24:19.869283+0000","last_unstale":"2023-11-24T06:24:20.904290+0000","last_undegraded":"2023-11-24T06:24:20.904290+0000","last_fullsized":"2023-11-24T06:24:20.904290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883379+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:19.883379+0000","last_peered":"2023-11-24T06:24:19.883379+0000","last_clean":"2023-11-24T06:24:19.883379+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:19.883379+0000","last_undegraded":"2023-11-24T06:24:19.883379+0000","last_fullsized":"2023-11-24T06:24:19.883379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979537+0000","last_change":"2023-11-24T06:24:19.869681+0000","last_active":"2023-11-24T06:24:20.979537+0000","last_peered":"2023-11-24T06:24:20.979537+0000","last_clean":"2023-11-24T06:24:20.979537+0000","last_became_active":"2023-11-24T06:24:19.869437+0000","last_became_peered":"2023-11-24T06:24:19.869437+0000","last_unstale":"2023-11-24T06:24:20.979537+0000","last_undegraded":"2023-11-24T06:24:20.979537+0000","last_fullsized":"2023-11-24T06:24:20.979537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879979+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:19.879979+0000","last_peered":"2023-11-24T06:24:19.879979+0000","last_clean":"2023-11-24T06:24:19.879979+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:19.879979+0000","last_undegraded":"2023-11-24T06:24:19.879979+0000","last_fullsized":"2023-11-24T06:24:19.879979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:29.087819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054525+0000","last_change":"2023-11-24T06:24:19.869851+0000","last_active":"2023-11-24T06:24:21.054525+0000","last_peered":"2023-11-24T06:24:21.054525+0000","last_clean":"2023-11-24T06:24:21.054525+0000","last_became_active":"2023-11-24T06:24:19.869550+0000","last_became_peered":"2023-11-24T06:24:19.869550+0000","last_unstale":"2023-11-24T06:24:21.054525+0000","last_undegraded":"2023-11-24T06:24:21.054525+0000","last_fullsized":"2023-11-24T06:24:21.054525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880299+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:19.880299+0000","last_peered":"2023-11-24T06:24:19.880299+0000","last_clean":"2023-11-24T06:24:19.880299+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:19.880299+0000","last_undegraded":"2023-11-24T06:24:19.880299+0000","last_fullsized":"2023-11-24T06:24:19.880299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054853+0000","last_change":"2023-11-24T06:24:19.870442+0000","last_active":"2023-11-24T06:24:21.054853+0000","last_peered":"2023-11-24T06:24:21.054853+0000","last_clean":"2023-11-24T06:24:21.054853+0000","last_became_active":"2023-11-24T06:24:19.870164+0000","last_became_peered":"2023-11-24T06:24:19.870164+0000","last_unstale":"2023-11-24T06:24:21.054853+0000","last_undegraded":"2023-11-24T06:24:21.054853+0000","last_fullsized":"2023-11-24T06:24:21.054853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883377+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:19.883377+0000","last_peered":"2023-11-24T06:24:19.883377+0000","last_clean":"2023-11-24T06:24:19.883377+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:19.883377+0000","last_undegraded":"2023-11-24T06:24:19.883377+0000","last_fullsized":"2023-11-24T06:24:19.883377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855381+0000","last_change":"2023-11-24T06:24:19.864863+0000","last_active":"2023-11-24T06:24:20.855381+0000","last_peered":"2023-11-24T06:24:20.855381+0000","last_clean":"2023-11-24T06:24:20.855381+0000","last_became_active":"2023-11-24T06:24:19.864621+0000","last_became_peered":"2023-11-24T06:24:19.864621+0000","last_unstale":"2023-11-24T06:24:20.855381+0000","last_undegraded":"2023-11-24T06:24:20.855381+0000","last_fullsized":"2023-11-24T06:24:20.855381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:51.049403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879363+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:19.879363+0000","last_peered":"2023-11-24T06:24:19.879363+0000","last_clean":"2023-11-24T06:24:19.879363+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:19.879363+0000","last_undegraded":"2023-11-24T06:24:19.879363+0000","last_fullsized":"2023-11-24T06:24:19.879363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979510+0000","last_change":"2023-11-24T06:24:19.866029+0000","last_active":"2023-11-24T06:24:20.979510+0000","last_peered":"2023-11-24T06:24:20.979510+0000","last_clean":"2023-11-24T06:24:20.979510+0000","last_became_active":"2023-11-24T06:24:19.865361+0000","last_became_peered":"2023-11-24T06:24:19.865361+0000","last_unstale":"2023-11-24T06:24:20.979510+0000","last_undegraded":"2023-11-24T06:24:20.979510+0000","last_fullsized":"2023-11-24T06:24:20.979510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852200+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:19.852200+0000","last_peered":"2023-11-24T06:24:19.852200+0000","last_clean":"2023-11-24T06:24:19.852200+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:19.852200+0000","last_undegraded":"2023-11-24T06:24:19.852200+0000","last_fullsized":"2023-11-24T06:24:19.852200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904187+0000","last_change":"2023-11-24T06:24:19.863134+0000","last_active":"2023-11-24T06:24:20.904187+0000","last_peered":"2023-11-24T06:24:20.904187+0000","last_clean":"2023-11-24T06:24:20.904187+0000","last_became_active":"2023-11-24T06:24:19.862679+0000","last_became_peered":"2023-11-24T06:24:19.862679+0000","last_unstale":"2023-11-24T06:24:20.904187+0000","last_undegraded":"2023-11-24T06:24:20.904187+0000","last_fullsized":"2023-11-24T06:24:20.904187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.862408+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:19.862408+0000","last_peered":"2023-11-24T06:24:19.862408+0000","last_clean":"2023-11-24T06:24:19.862408+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:19.862408+0000","last_undegraded":"2023-11-24T06:24:19.862408+0000","last_fullsized":"2023-11-24T06:24:19.862408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853050+0000","last_change":"2023-11-24T06:24:19.864439+0000","last_active":"2023-11-24T06:24:20.853050+0000","last_peered":"2023-11-24T06:24:20.853050+0000","last_clean":"2023-11-24T06:24:20.853050+0000","last_became_active":"2023-11-24T06:24:19.864128+0000","last_became_peered":"2023-11-24T06:24:19.864128+0000","last_unstale":"2023-11-24T06:24:20.853050+0000","last_undegraded":"2023-11-24T06:24:20.853050+0000","last_fullsized":"2023-11-24T06:24:20.853050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860879+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:19.860879+0000","last_peered":"2023-11-24T06:24:19.860879+0000","last_clean":"2023-11-24T06:24:19.860879+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:19.860879+0000","last_undegraded":"2023-11-24T06:24:19.860879+0000","last_fullsized":"2023-11-24T06:24:19.860879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054843+0000","last_change":"2023-11-24T06:24:19.874520+0000","last_active":"2023-11-24T06:24:21.054843+0000","last_peered":"2023-11-24T06:24:21.054843+0000","last_clean":"2023-11-24T06:24:21.054843+0000","last_became_active":"2023-11-24T06:24:19.872481+0000","last_became_peered":"2023-11-24T06:24:19.872481+0000","last_unstale":"2023-11-24T06:24:21.054843+0000","last_undegraded":"2023-11-24T06:24:21.054843+0000","last_fullsized":"2023-11-24T06:24:21.054843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.865095+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:19.865095+0000","last_peered":"2023-11-24T06:24:19.865095+0000","last_clean":"2023-11-24T06:24:19.865095+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:19.865095+0000","last_undegraded":"2023-11-24T06:24:19.865095+0000","last_fullsized":"2023-11-24T06:24:19.865095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:51.886938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979608+0000","last_change":"2023-11-24T06:24:19.867545+0000","last_active":"2023-11-24T06:24:20.979608+0000","last_peered":"2023-11-24T06:24:20.979608+0000","last_clean":"2023-11-24T06:24:20.979608+0000","last_became_active":"2023-11-24T06:24:19.867313+0000","last_became_peered":"2023-11-24T06:24:19.867313+0000","last_unstale":"2023-11-24T06:24:20.979608+0000","last_undegraded":"2023-11-24T06:24:20.979608+0000","last_fullsized":"2023-11-24T06:24:20.979608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885062+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:19.885062+0000","last_peered":"2023-11-24T06:24:19.885062+0000","last_clean":"2023-11-24T06:24:19.885062+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:19.885062+0000","last_undegraded":"2023-11-24T06:24:19.885062+0000","last_fullsized":"2023-11-24T06:24:19.885062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:08.350203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855876+0000","last_change":"2023-11-24T06:24:19.864361+0000","last_active":"2023-11-24T06:24:20.855876+0000","last_peered":"2023-11-24T06:24:20.855876+0000","last_clean":"2023-11-24T06:24:20.855876+0000","last_became_active":"2023-11-24T06:24:19.864077+0000","last_became_peered":"2023-11-24T06:24:19.864077+0000","last_unstale":"2023-11-24T06:24:20.855876+0000","last_undegraded":"2023-11-24T06:24:20.855876+0000","last_fullsized":"2023-11-24T06:24:20.855876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885718+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:19.885718+0000","last_peered":"2023-11-24T06:24:19.885718+0000","last_clean":"2023-11-24T06:24:19.885718+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:19.885718+0000","last_undegraded":"2023-11-24T06:24:19.885718+0000","last_fullsized":"2023-11-24T06:24:19.885718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854301+0000","last_change":"2023-11-24T06:24:19.858071+0000","last_active":"2023-11-24T06:24:20.854301+0000","last_peered":"2023-11-24T06:24:20.854301+0000","last_clean":"2023-11-24T06:24:20.854301+0000","last_became_active":"2023-11-24T06:24:19.857803+0000","last_became_peered":"2023-11-24T06:24:19.857803+0000","last_unstale":"2023-11-24T06:24:20.854301+0000","last_undegraded":"2023-11-24T06:24:20.854301+0000","last_fullsized":"2023-11-24T06:24:20.854301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.882906+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:19.882906+0000","last_peered":"2023-11-24T06:24:19.882906+0000","last_clean":"2023-11-24T06:24:19.882906+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:19.882906+0000","last_undegraded":"2023-11-24T06:24:19.882906+0000","last_fullsized":"2023-11-24T06:24:19.882906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979188+0000","last_change":"2023-11-24T06:24:19.870356+0000","last_active":"2023-11-24T06:24:20.979188+0000","last_peered":"2023-11-24T06:24:20.979188+0000","last_clean":"2023-11-24T06:24:20.979188+0000","last_became_active":"2023-11-24T06:24:19.870057+0000","last_became_peered":"2023-11-24T06:24:19.870057+0000","last_unstale":"2023-11-24T06:24:20.979188+0000","last_undegraded":"2023-11-24T06:24:20.979188+0000","last_fullsized":"2023-11-24T06:24:20.979188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:30.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864248+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:19.864248+0000","last_peered":"2023-11-24T06:24:19.864248+0000","last_clean":"2023-11-24T06:24:19.864248+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:19.864248+0000","last_undegraded":"2023-11-24T06:24:19.864248+0000","last_fullsized":"2023-11-24T06:24:19.864248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854429+0000","last_change":"2023-11-24T06:24:19.863989+0000","last_active":"2023-11-24T06:24:20.854429+0000","last_peered":"2023-11-24T06:24:20.854429+0000","last_clean":"2023-11-24T06:24:20.854429+0000","last_became_active":"2023-11-24T06:24:19.861502+0000","last_became_peered":"2023-11-24T06:24:19.861502+0000","last_unstale":"2023-11-24T06:24:20.854429+0000","last_undegraded":"2023-11-24T06:24:20.854429+0000","last_fullsized":"2023-11-24T06:24:20.854429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860367+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:19.860367+0000","last_peered":"2023-11-24T06:24:19.860367+0000","last_clean":"2023-11-24T06:24:19.860367+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:19.860367+0000","last_undegraded":"2023-11-24T06:24:19.860367+0000","last_fullsized":"2023-11-24T06:24:19.860367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553322+0000","last_change":"2023-11-24T06:24:19.873616+0000","last_active":"2023-11-24T06:24:21.553322+0000","last_peered":"2023-11-24T06:24:21.553322+0000","last_clean":"2023-11-24T06:24:21.553322+0000","last_became_active":"2023-11-24T06:24:19.873308+0000","last_became_peered":"2023-11-24T06:24:19.873308+0000","last_unstale":"2023-11-24T06:24:21.553322+0000","last_undegraded":"2023-11-24T06:24:21.553322+0000","last_fullsized":"2023-11-24T06:24:21.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.886102+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:19.886102+0000","last_peered":"2023-11-24T06:24:19.886102+0000","last_clean":"2023-11-24T06:24:19.886102+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:19.886102+0000","last_undegraded":"2023-11-24T06:24:19.886102+0000","last_fullsized":"2023-11-24T06:24:19.886102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854591+0000","last_change":"2023-11-24T06:24:19.865497+0000","last_active":"2023-11-24T06:24:20.854591+0000","last_peered":"2023-11-24T06:24:20.854591+0000","last_clean":"2023-11-24T06:24:20.854591+0000","last_became_active":"2023-11-24T06:24:19.863266+0000","last_became_peered":"2023-11-24T06:24:19.863266+0000","last_unstale":"2023-11-24T06:24:20.854591+0000","last_undegraded":"2023-11-24T06:24:20.854591+0000","last_fullsized":"2023-11-24T06:24:20.854591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883170+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:19.883170+0000","last_peered":"2023-11-24T06:24:19.883170+0000","last_clean":"2023-11-24T06:24:19.883170+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:19.883170+0000","last_undegraded":"2023-11-24T06:24:19.883170+0000","last_fullsized":"2023-11-24T06:24:19.883170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854478+0000","last_change":"2023-11-24T06:24:19.862689+0000","last_active":"2023-11-24T06:24:20.854478+0000","last_peered":"2023-11-24T06:24:20.854478+0000","last_clean":"2023-11-24T06:24:20.854478+0000","last_became_active":"2023-11-24T06:24:19.862254+0000","last_became_peered":"2023-11-24T06:24:19.862254+0000","last_unstale":"2023-11-24T06:24:20.854478+0000","last_undegraded":"2023-11-24T06:24:20.854478+0000","last_fullsized":"2023-11-24T06:24:20.854478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860268+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:19.860268+0000","last_peered":"2023-11-24T06:24:19.860268+0000","last_clean":"2023-11-24T06:24:19.860268+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:19.860268+0000","last_undegraded":"2023-11-24T06:24:19.860268+0000","last_fullsized":"2023-11-24T06:24:19.860268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054699+0000","last_change":"2023-11-24T06:24:19.876750+0000","last_active":"2023-11-24T06:24:21.054699+0000","last_peered":"2023-11-24T06:24:21.054699+0000","last_clean":"2023-11-24T06:24:21.054699+0000","last_became_active":"2023-11-24T06:24:19.876463+0000","last_became_peered":"2023-11-24T06:24:19.876463+0000","last_unstale":"2023-11-24T06:24:21.054699+0000","last_undegraded":"2023-11-24T06:24:21.054699+0000","last_fullsized":"2023-11-24T06:24:21.054699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864116+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:19.864116+0000","last_peered":"2023-11-24T06:24:19.864116+0000","last_clean":"2023-11-24T06:24:19.864116+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:19.864116+0000","last_undegraded":"2023-11-24T06:24:19.864116+0000","last_fullsized":"2023-11-24T06:24:19.864116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855678+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:20.855678+0000","last_peered":"2023-11-24T06:24:20.855678+0000","last_clean":"2023-11-24T06:24:20.855678+0000","last_became_active":"2023-11-24T06:24:19.866108+0000","last_became_peered":"2023-11-24T06:24:19.866108+0000","last_unstale":"2023-11-24T06:24:20.855678+0000","last_undegraded":"2023-11-24T06:24:20.855678+0000","last_fullsized":"2023-11-24T06:24:20.855678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:03.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852935+0000","last_change":"2023-11-24T06:24:19.865135+0000","last_active":"2023-11-24T06:24:20.852935+0000","last_peered":"2023-11-24T06:24:20.852935+0000","last_clean":"2023-11-24T06:24:20.852935+0000","last_became_active":"2023-11-24T06:24:19.864837+0000","last_became_peered":"2023-11-24T06:24:19.864837+0000","last_unstale":"2023-11-24T06:24:20.852935+0000","last_undegraded":"2023-11-24T06:24:20.852935+0000","last_fullsized":"2023-11-24T06:24:20.852935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857378+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:19.857378+0000","last_peered":"2023-11-24T06:24:19.857378+0000","last_clean":"2023-11-24T06:24:19.857378+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:19.857378+0000","last_undegraded":"2023-11-24T06:24:19.857378+0000","last_fullsized":"2023-11-24T06:24:19.857378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904182+0000","last_change":"2023-11-24T06:24:19.871367+0000","last_active":"2023-11-24T06:24:20.904182+0000","last_peered":"2023-11-24T06:24:20.904182+0000","last_clean":"2023-11-24T06:24:20.904182+0000","last_became_active":"2023-11-24T06:24:19.871058+0000","last_became_peered":"2023-11-24T06:24:19.871058+0000","last_unstale":"2023-11-24T06:24:20.904182+0000","last_undegraded":"2023-11-24T06:24:20.904182+0000","last_fullsized":"2023-11-24T06:24:20.904182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.858425+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:19.858425+0000","last_peered":"2023-11-24T06:24:19.858425+0000","last_clean":"2023-11-24T06:24:19.858425+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:19.858425+0000","last_undegraded":"2023-11-24T06:24:19.858425+0000","last_fullsized":"2023-11-24T06:24:19.858425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852316+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:19.852316+0000","last_peered":"2023-11-24T06:24:19.852316+0000","last_clean":"2023-11-24T06:24:19.852316+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:19.852316+0000","last_undegraded":"2023-11-24T06:24:19.852316+0000","last_fullsized":"2023-11-24T06:24:19.852316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979177+0000","last_change":"2023-11-24T06:24:19.864862+0000","last_active":"2023-11-24T06:24:20.979177+0000","last_peered":"2023-11-24T06:24:20.979177+0000","last_clean":"2023-11-24T06:24:20.979177+0000","last_became_active":"2023-11-24T06:24:19.863541+0000","last_became_peered":"2023-11-24T06:24:19.863541+0000","last_unstale":"2023-11-24T06:24:20.979177+0000","last_undegraded":"2023-11-24T06:24:20.979177+0000","last_fullsized":"2023-11-24T06:24:20.979177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852544+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:19.852544+0000","last_peered":"2023-11-24T06:24:19.852544+0000","last_clean":"2023-11-24T06:24:19.852544+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:19.852544+0000","last_undegraded":"2023-11-24T06:24:19.852544+0000","last_fullsized":"2023-11-24T06:24:19.852544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.853979+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:19.853979+0000","last_peered":"2023-11-24T06:24:19.853979+0000","last_clean":"2023-11-24T06:24:19.853979+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:19.853979+0000","last_undegraded":"2023-11-24T06:24:19.853979+0000","last_fullsized":"2023-11-24T06:24:19.853979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:48.736229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853074+0000","last_change":"2023-11-24T06:24:19.870498+0000","last_active":"2023-11-24T06:24:20.853074+0000","last_peered":"2023-11-24T06:24:20.853074+0000","last_clean":"2023-11-24T06:24:20.853074+0000","last_became_active":"2023-11-24T06:24:19.870199+0000","last_became_peered":"2023-11-24T06:24:19.870199+0000","last_unstale":"2023-11-24T06:24:20.853074+0000","last_undegraded":"2023-11-24T06:24:20.853074+0000","last_fullsized":"2023-11-24T06:24:20.853074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860174+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:19.860174+0000","last_peered":"2023-11-24T06:24:19.860174+0000","last_clean":"2023-11-24T06:24:19.860174+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:19.860174+0000","last_undegraded":"2023-11-24T06:24:19.860174+0000","last_fullsized":"2023-11-24T06:24:19.860174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904362+0000","last_change":"2023-11-24T06:24:19.869468+0000","last_active":"2023-11-24T06:24:20.904362+0000","last_peered":"2023-11-24T06:24:20.904362+0000","last_clean":"2023-11-24T06:24:20.904362+0000","last_became_active":"2023-11-24T06:24:19.869176+0000","last_became_peered":"2023-11-24T06:24:19.869176+0000","last_unstale":"2023-11-24T06:24:20.904362+0000","last_undegraded":"2023-11-24T06:24:20.904362+0000","last_fullsized":"2023-11-24T06:24:20.904362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880276+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:19.880276+0000","last_peered":"2023-11-24T06:24:19.880276+0000","last_clean":"2023-11-24T06:24:19.880276+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:19.880276+0000","last_undegraded":"2023-11-24T06:24:19.880276+0000","last_fullsized":"2023-11-24T06:24:19.880276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979489+0000","last_change":"2023-11-24T06:24:19.868306+0000","last_active":"2023-11-24T06:24:20.979489+0000","last_peered":"2023-11-24T06:24:20.979489+0000","last_clean":"2023-11-24T06:24:20.979489+0000","last_became_active":"2023-11-24T06:24:19.867148+0000","last_became_peered":"2023-11-24T06:24:19.867148+0000","last_unstale":"2023-11-24T06:24:20.979489+0000","last_undegraded":"2023-11-24T06:24:20.979489+0000","last_fullsized":"2023-11-24T06:24:20.979489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852405+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:19.852405+0000","last_peered":"2023-11-24T06:24:19.852405+0000","last_clean":"2023-11-24T06:24:19.852405+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:19.852405+0000","last_undegraded":"2023-11-24T06:24:19.852405+0000","last_fullsized":"2023-11-24T06:24:19.852405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855451+0000","last_change":"2023-11-24T06:24:19.862616+0000","last_active":"2023-11-24T06:24:20.855451+0000","last_peered":"2023-11-24T06:24:20.855451+0000","last_clean":"2023-11-24T06:24:20.855451+0000","last_became_active":"2023-11-24T06:24:19.862284+0000","last_became_peered":"2023-11-24T06:24:19.862284+0000","last_unstale":"2023-11-24T06:24:20.855451+0000","last_undegraded":"2023-11-24T06:24:20.855451+0000","last_fullsized":"2023-11-24T06:24:20.855451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:46.608575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859914+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:19.859914+0000","last_peered":"2023-11-24T06:24:19.859914+0000","last_clean":"2023-11-24T06:24:19.859914+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:19.859914+0000","last_undegraded":"2023-11-24T06:24:19.859914+0000","last_fullsized":"2023-11-24T06:24:19.859914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904102+0000","last_change":"2023-11-24T06:24:19.866929+0000","last_active":"2023-11-24T06:24:20.904102+0000","last_peered":"2023-11-24T06:24:20.904102+0000","last_clean":"2023-11-24T06:24:20.904102+0000","last_became_active":"2023-11-24T06:24:19.866522+0000","last_became_peered":"2023-11-24T06:24:19.866522+0000","last_unstale":"2023-11-24T06:24:20.904102+0000","last_undegraded":"2023-11-24T06:24:20.904102+0000","last_fullsized":"2023-11-24T06:24:20.904102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859599+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:19.859599+0000","last_peered":"2023-11-24T06:24:19.859599+0000","last_clean":"2023-11-24T06:24:19.859599+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:19.859599+0000","last_undegraded":"2023-11-24T06:24:19.859599+0000","last_fullsized":"2023-11-24T06:24:19.859599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904434+0000","last_change":"2023-11-24T06:24:19.870623+0000","last_active":"2023-11-24T06:24:20.904434+0000","last_peered":"2023-11-24T06:24:20.904434+0000","last_clean":"2023-11-24T06:24:20.904434+0000","last_became_active":"2023-11-24T06:24:19.869946+0000","last_became_peered":"2023-11-24T06:24:19.869946+0000","last_unstale":"2023-11-24T06:24:20.904434+0000","last_undegraded":"2023-11-24T06:24:20.904434+0000","last_fullsized":"2023-11-24T06:24:20.904434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879779+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:19.879779+0000","last_peered":"2023-11-24T06:24:19.879779+0000","last_clean":"2023-11-24T06:24:19.879779+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:19.879779+0000","last_undegraded":"2023-11-24T06:24:19.879779+0000","last_fullsized":"2023-11-24T06:24:19.879779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:08.503960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553085+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:21.553085+0000","last_peered":"2023-11-24T06:24:21.553085+0000","last_clean":"2023-11-24T06:24:21.553085+0000","last_became_active":"2023-11-24T06:24:19.866115+0000","last_became_peered":"2023-11-24T06:24:19.866115+0000","last_unstale":"2023-11-24T06:24:21.553085+0000","last_undegraded":"2023-11-24T06:24:21.553085+0000","last_fullsized":"2023-11-24T06:24:21.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857112+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:19.857112+0000","last_peered":"2023-11-24T06:24:19.857112+0000","last_clean":"2023-11-24T06:24:19.857112+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:19.857112+0000","last_undegraded":"2023-11-24T06:24:19.857112+0000","last_fullsized":"2023-11-24T06:24:19.857112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854430+0000","last_change":"2023-11-24T06:24:19.867560+0000","last_active":"2023-11-24T06:24:20.854430+0000","last_peered":"2023-11-24T06:24:20.854430+0000","last_clean":"2023-11-24T06:24:20.854430+0000","last_became_active":"2023-11-24T06:24:19.867277+0000","last_became_peered":"2023-11-24T06:24:19.867277+0000","last_unstale":"2023-11-24T06:24:20.854430+0000","last_undegraded":"2023-11-24T06:24:20.854430+0000","last_fullsized":"2023-11-24T06:24:20.854430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885158+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:19.885158+0000","last_peered":"2023-11-24T06:24:19.885158+0000","last_clean":"2023-11-24T06:24:19.885158+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:19.885158+0000","last_undegraded":"2023-11-24T06:24:19.885158+0000","last_fullsized":"2023-11-24T06:24:19.885158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054548+0000","last_change":"2023-11-24T06:24:19.875426+0000","last_active":"2023-11-24T06:24:21.054548+0000","last_peered":"2023-11-24T06:24:21.054548+0000","last_clean":"2023-11-24T06:24:21.054548+0000","last_became_active":"2023-11-24T06:24:19.875167+0000","last_became_peered":"2023-11-24T06:24:19.875167+0000","last_unstale":"2023-11-24T06:24:21.054548+0000","last_undegraded":"2023-11-24T06:24:21.054548+0000","last_fullsized":"2023-11-24T06:24:21.054548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.884010+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:19.884010+0000","last_peered":"2023-11-24T06:24:19.884010+0000","last_clean":"2023-11-24T06:24:19.884010+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:19.884010+0000","last_undegraded":"2023-11-24T06:24:19.884010+0000","last_fullsized":"2023-11-24T06:24:19.884010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:50.045740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854899+0000","last_change":"2023-11-24T06:24:19.866835+0000","last_active":"2023-11-24T06:24:20.854899+0000","last_peered":"2023-11-24T06:24:20.854899+0000","last_clean":"2023-11-24T06:24:20.854899+0000","last_became_active":"2023-11-24T06:24:19.866476+0000","last_became_peered":"2023-11-24T06:24:19.866476+0000","last_unstale":"2023-11-24T06:24:20.854899+0000","last_undegraded":"2023-11-24T06:24:20.854899+0000","last_fullsized":"2023-11-24T06:24:20.854899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:20.926427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553269+0000","last_change":"2023-11-24T06:24:19.873151+0000","last_active":"2023-11-24T06:24:21.553269+0000","last_peered":"2023-11-24T06:24:21.553269+0000","last_clean":"2023-11-24T06:24:21.553269+0000","last_became_active":"2023-11-24T06:24:19.872848+0000","last_became_peered":"2023-11-24T06:24:19.872848+0000","last_unstale":"2023-11-24T06:24:21.553269+0000","last_undegraded":"2023-11-24T06:24:21.553269+0000","last_fullsized":"2023-11-24T06:24:21.553269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553077+0000","last_change":"2023-11-24T06:24:19.866025+0000","last_active":"2023-11-24T06:24:21.553077+0000","last_peered":"2023-11-24T06:24:21.553077+0000","last_clean":"2023-11-24T06:24:21.553077+0000","last_became_active":"2023-11-24T06:24:19.865501+0000","last_became_peered":"2023-11-24T06:24:19.865501+0000","last_unstale":"2023-11-24T06:24:21.553077+0000","last_undegraded":"2023-11-24T06:24:21.553077+0000","last_fullsized":"2023-11-24T06:24:21.553077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054789+0000","last_change":"2023-11-24T06:24:19.876131+0000","last_active":"2023-11-24T06:24:21.054789+0000","last_peered":"2023-11-24T06:24:21.054789+0000","last_clean":"2023-11-24T06:24:21.054789+0000","last_became_active":"2023-11-24T06:24:19.875876+0000","last_became_peered":"2023-11-24T06:24:19.875876+0000","last_unstale":"2023-11-24T06:24:21.054789+0000","last_undegraded":"2023-11-24T06:24:21.054789+0000","last_fullsized":"2023-11-24T06:24:21.054789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979358+0000","last_change":"2023-11-24T06:24:19.865439+0000","last_active":"2023-11-24T06:24:20.979358+0000","last_peered":"2023-11-24T06:24:20.979358+0000","last_clean":"2023-11-24T06:24:20.979358+0000","last_became_active":"2023-11-24T06:24:19.865124+0000","last_became_peered":"2023-11-24T06:24:19.865124+0000","last_unstale":"2023-11-24T06:24:20.979358+0000","last_undegraded":"2023-11-24T06:24:20.979358+0000","last_fullsized":"2023-11-24T06:24:20.979358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904352+0000","last_change":"2023-11-24T06:24:19.865010+0000","last_active":"2023-11-24T06:24:20.904352+0000","last_peered":"2023-11-24T06:24:20.904352+0000","last_clean":"2023-11-24T06:24:20.904352+0000","last_became_active":"2023-11-24T06:24:19.864730+0000","last_became_peered":"2023-11-24T06:24:19.864730+0000","last_unstale":"2023-11-24T06:24:20.904352+0000","last_undegraded":"2023-11-24T06:24:20.904352+0000","last_fullsized":"2023-11-24T06:24:20.904352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853108+0000","last_change":"2023-11-24T06:24:19.873933+0000","last_active":"2023-11-24T06:24:20.853108+0000","last_peered":"2023-11-24T06:24:20.853108+0000","last_clean":"2023-11-24T06:24:20.853108+0000","last_became_active":"2023-11-24T06:24:19.873679+0000","last_became_peered":"2023-11-24T06:24:19.873679+0000","last_unstale":"2023-11-24T06:24:20.853108+0000","last_undegraded":"2023-11-24T06:24:20.853108+0000","last_fullsized":"2023-11-24T06:24:20.853108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553458+0000","last_change":"2023-11-24T06:24:19.873987+0000","last_active":"2023-11-24T06:24:21.553458+0000","last_peered":"2023-11-24T06:24:21.553458+0000","last_clean":"2023-11-24T06:24:21.553458+0000","last_became_active":"2023-11-24T06:24:19.871835+0000","last_became_peered":"2023-11-24T06:24:19.871835+0000","last_unstale":"2023-11-24T06:24:21.553458+0000","last_undegraded":"2023-11-24T06:24:21.553458+0000","last_fullsized":"2023-11-24T06:24:21.553458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904255+0000","last_change":"2023-11-24T06:24:19.863300+0000","last_active":"2023-11-24T06:24:20.904255+0000","last_peered":"2023-11-24T06:24:20.904255+0000","last_clean":"2023-11-24T06:24:20.904255+0000","last_became_active":"2023-11-24T06:24:19.861918+0000","last_became_peered":"2023-11-24T06:24:19.861918+0000","last_unstale":"2023-11-24T06:24:20.904255+0000","last_undegraded":"2023-11-24T06:24:20.904255+0000","last_fullsized":"2023-11-24T06:24:20.904255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054464+0000","last_change":"2023-11-24T06:24:19.878153+0000","last_active":"2023-11-24T06:24:21.054464+0000","last_peered":"2023-11-24T06:24:21.054464+0000","last_clean":"2023-11-24T06:24:21.054464+0000","last_became_active":"2023-11-24T06:24:19.876482+0000","last_became_peered":"2023-11-24T06:24:19.876482+0000","last_unstale":"2023-11-24T06:24:21.054464+0000","last_undegraded":"2023-11-24T06:24:21.054464+0000","last_fullsized":"2023-11-24T06:24:21.054464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904064+0000","last_change":"2023-11-24T06:24:19.862468+0000","last_active":"2023-11-24T06:24:20.904064+0000","last_peered":"2023-11-24T06:24:20.904064+0000","last_clean":"2023-11-24T06:24:20.904064+0000","last_became_active":"2023-11-24T06:24:19.862178+0000","last_became_peered":"2023-11-24T06:24:19.862178+0000","last_unstale":"2023-11-24T06:24:20.904064+0000","last_undegraded":"2023-11-24T06:24:20.904064+0000","last_fullsized":"2023-11-24T06:24:20.904064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054544+0000","last_change":"2023-11-24T06:24:19.871577+0000","last_active":"2023-11-24T06:24:21.054544+0000","last_peered":"2023-11-24T06:24:21.054544+0000","last_clean":"2023-11-24T06:24:21.054544+0000","last_became_active":"2023-11-24T06:24:19.871279+0000","last_became_peered":"2023-11-24T06:24:19.871279+0000","last_unstale":"2023-11-24T06:24:21.054544+0000","last_undegraded":"2023-11-24T06:24:21.054544+0000","last_fullsized":"2023-11-24T06:24:21.054544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979353+0000","last_change":"2023-11-24T06:24:19.872608+0000","last_active":"2023-11-24T06:24:20.979353+0000","last_peered":"2023-11-24T06:24:20.979353+0000","last_clean":"2023-11-24T06:24:20.979353+0000","last_became_active":"2023-11-24T06:24:19.872343+0000","last_became_peered":"2023-11-24T06:24:19.872343+0000","last_unstale":"2023-11-24T06:24:20.979353+0000","last_undegraded":"2023-11-24T06:24:20.979353+0000","last_fullsized":"2023-11-24T06:24:20.979353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553301+0000","last_change":"2023-11-24T06:24:19.868776+0000","last_active":"2023-11-24T06:24:21.553301+0000","last_peered":"2023-11-24T06:24:21.553301+0000","last_clean":"2023-11-24T06:24:21.553301+0000","last_became_active":"2023-11-24T06:24:19.868335+0000","last_became_peered":"2023-11-24T06:24:19.868335+0000","last_unstale":"2023-11-24T06:24:21.553301+0000","last_undegraded":"2023-11-24T06:24:21.553301+0000","last_fullsized":"2023-11-24T06:24:21.553301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:28.886 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:24:28.886 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:24:28.886 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:24:28.886 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:24:28.887 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:24:29.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.038+0000 7f50ec886700 1 Processor -- start 2023-11-24T06:24:29.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.038+0000 7f50ec886700 1 -- start start 2023-11-24T06:24:29.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.039+0000 7f50ec886700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 0x7f50e81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.039+0000 7f50ec886700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f50e80d1c70 0x7f50e80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.039+0000 7f50ec886700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80b22d0 con 0x7f50e80b3370 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.039+0000 7f50ec886700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e80b2f50 con 0x7f50e80d1c70 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.039+0000 7f50ec886700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e80d2640 con 0x7f50e815bdd0 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f50e80d1c70 0x7f50e80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e5d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f50e80b3370 0x7f50e80b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53584/0 (socket says 172.21.15.114:53584) 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e5d9b700 1 -- 172.21.15.114:0/1967641738 learned_addr learned my addr 172.21.15.114:0/1967641738 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e659c700 1 --2- 172.21.15.114:0/1967641738 >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 0x7f50e81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 -- 172.21.15.114:0/1967641738 >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 msgr2=0x7f50e81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 --2- 172.21.15.114:0/1967641738 >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 0x7f50e81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 -- 172.21.15.114:0/1967641738 >> v1:172.21.15.114:6789/0 conn(0x7f50e80b3370 legacy=0x7f50e80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e559a700 1 -- 172.21.15.114:0/1967641738 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25919548 0 0) 0x7f50e80b22d0 con 0x7f50e80b3370 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 -- 172.21.15.114:0/1967641738 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80b25b0 con 0x7f50e80d1c70 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e659c700 1 --2- 172.21.15.114:0/1967641738 >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 0x7f50e81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e6d9d700 1 --2- 172.21.15.114:0/1967641738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80d1c70 0x7f50e80d2100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f50e806ec10 tx=0x7f50e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff9efdaa3ffdc8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e559a700 1 -- 172.21.15.114:0/1967641738 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000e070 con 0x7f50e80d1c70 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e559a700 1 -- 172.21.15.114:0/1967641738 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000bbd0 con 0x7f50e80d1c70 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.040+0000 7f50e559a700 1 -- 172.21.15.114:0/1967641738 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0012430 con 0x7f50e80d1c70 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 -- 172.21.15.114:0/1967641738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80d1c70 msgr2=0x7f50e80d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 --2- 172.21.15.114:0/1967641738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80d1c70 0x7f50e80d2100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f50e806ec10 tx=0x7f50e000b330 comp rx=0 tx=0).stop 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 -- 172.21.15.114:0/1967641738 shutdown_connections 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 --2- 172.21.15.114:0/1967641738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80d1c70 0x7f50e80d2100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 --2- 172.21.15.114:0/1967641738 >> v2:172.21.15.172:3300/0 conn(0x7f50e815bdd0 0x7f50e81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 -- 172.21.15.114:0/1967641738 >> 172.21.15.114:0/1967641738 conn(0x7f50e80731a0 msgr2=0x7f50e809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 -- 172.21.15.114:0/1967641738 shutdown_connections 2023-11-24T06:24:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.041+0000 7f50ec886700 1 -- 172.21.15.114:0/1967641738 wait complete. 2023-11-24T06:24:29.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.042+0000 7f50ec886700 1 Processor -- start 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.042+0000 7f50ec886700 1 -- start start 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50e80d1c70 0x7f50e806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50e815bdd0 0x7f50e80b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e81578a0 con 0x7f50e80b3370 2023-11-24T06:24:29.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e806d020 con 0x7f50e80d1c70 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50ec886700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80ce620 con 0x7f50e815bdd0 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.043+0000 7f50e659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37882/0 (socket says 172.21.15.114:37882) 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 -- 172.21.15.114:0/1921298881 learned_addr learned my addr 172.21.15.114:0/1921298881 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 -- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50e815bdd0 msgr2=0x7f50e80b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e5d9b700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50e80d1c70 0x7f50e806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50e815bdd0 0x7f50e80b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 -- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50e80d1c70 msgr2=0x7f50e806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50e80d1c70 0x7f50e806cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 -- 172.21.15.114:0/1921298881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009a70 con 0x7f50e80b3370 2023-11-24T06:24:29.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50e659c700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00e810 tx=0x7f50dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00c020 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc004250 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e80cd6d0 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0036a0 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.044+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e80ceba0 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.045+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e8067f60 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.045+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50dc003800 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.046+0000 7f50d6ffd700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 0x7f50c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.046+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50dc089440 con 0x7f50e80b3370 2023-11-24T06:24:29.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.046+0000 7f50e5d9b700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 0x7f50c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.046+0000 7f50e5d9b700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 0x7f50c806d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50e80cfae0 tx=0x7f50d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.050+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50dc054470 con 0x7f50e80b3370 2023-11-24T06:24:29.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.298+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f50e80b7c60 con 0x7f50e80b3370 2023-11-24T06:24:29.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.301+0000 7f50d6ffd700 1 -- 172.21.15.114:0/1921298881 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171035 (secure 0 0 0) 0x7f50dc057a90 con 0x7f50e80b3370 2023-11-24T06:24:29.307 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 msgr2=0x7f50c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 0x7f50c806d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50e80cfae0 tx=0x7f50d8009040 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 msgr2=0x7f50e8157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00e810 tx=0x7f50dc00ebd0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 shutdown_connections 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f50c806aed0 0x7f50c806d390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50e815bdd0 0x7f50e80b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50e80b3370 0x7f50e8157360 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 --2- 172.21.15.114:0/1921298881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50e80d1c70 0x7f50e806cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 >> 172.21.15.114:0/1921298881 conn(0x7f50e80731a0 msgr2=0x7f50e80b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 shutdown_connections 2023-11-24T06:24:29.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.304+0000 7f50ec886700 1 -- 172.21.15.114:0/1921298881 wait complete. 2023-11-24T06:24:29.319 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":407740377},{"type":"v1","addr":"172.21.15.114:6833","nonce":407740377}]},"active_addr":"172.21.15.114:6833/407740377","active_change":"2023-11-24T06:23:57.698378+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1511706394}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":248249531}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1924452980}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2554539951}]}]} 2023-11-24T06:24:29.323 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:24:29.323 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:24:29.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:29.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.441+0000 7fa47015a700 1 Processor -- start 2023-11-24T06:24:29.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.441+0000 7fa47015a700 1 -- start start 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa47015a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa4680b3370 0x7fa4681604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa47015a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 0x7fa4680d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa47015a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4680b22d0 con 0x7fa46815bdd0 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa47015a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4680b2f50 con 0x7fa4680b3370 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa47015a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680d2640 con 0x7fa4680d1c70 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa4680b3370 0x7fa4681604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa4680b3370 0x7fa4681604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37892/0 (socket says 172.21.15.114:37892) 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 -- 172.21.15.114:0/3988235873 learned_addr learned my addr 172.21.15.114:0/3988235873 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46e6f7700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 0x7fa4680d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 -- 172.21.15.114:0/3988235873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 msgr2=0x7fa4680d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 0x7fa4680d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 -- 172.21.15.114:0/3988235873 >> v1:172.21.15.172:6789/0 conn(0x7fa46815bdd0 legacy=0x7fa4680b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 -- 172.21.15.114:0/3988235873 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4680b25b0 con 0x7fa4680b3370 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46e6f7700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 0x7fa4680d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.442+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4681604e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4580098f0 tx=0x7fa458009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440ab724a6d1ea71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa46cef4700 1 -- 172.21.15.114:0/3988235873 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800b8a0 con 0x7fa4680b3370 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa46cef4700 1 -- 172.21.15.114:0/3988235873 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45800ba00 con 0x7fa4680b3370 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa46cef4700 1 -- 172.21.15.114:0/3988235873 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800bd90 con 0x7fa4680b3370 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 -- 172.21.15.114:0/3988235873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 msgr2=0x7fa4681604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4681604e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa4580098f0 tx=0x7fa458009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 -- 172.21.15.114:0/3988235873 shutdown_connections 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4680d1c70 0x7fa4680d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 --2- 172.21.15.114:0/3988235873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4681604e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 -- 172.21.15.114:0/3988235873 >> 172.21.15.114:0/3988235873 conn(0x7fa4680731a0 msgr2=0x7fa46809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 -- 172.21.15.114:0/3988235873 shutdown_connections 2023-11-24T06:24:29.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.443+0000 7fa47015a700 1 -- 172.21.15.114:0/3988235873 wait complete. 2023-11-24T06:24:29.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.445+0000 7fa47015a700 1 Processor -- start 2023-11-24T06:24:29.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.445+0000 7fa47015a700 1 -- start start 2023-11-24T06:24:29.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 0x7fa468156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 0x7fa4680ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680cde90 con 0x7fa4680b3370 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ce7f0 con 0x7fa4680d1c70 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ccfc0 con 0x7fa46815bdd0 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37896/0 (socket says 172.21.15.114:37896) 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 -- 172.21.15.114:0/3213271364 learned_addr learned my addr 172.21.15.114:0/3213271364 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46e6f7700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 0x7fa4680ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 0x7fa468156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 -- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 msgr2=0x7fa4680ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 0x7fa4680ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 -- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 msgr2=0x7fa468156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 0x7fa468156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 -- 172.21.15.114:0/3213271364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009550 con 0x7fa4680b3370 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 0x7fa468156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46e6f7700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 0x7fa4680ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa46def6700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c450 tx=0x7fa46400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4640052d0 con 0x7fa4680b3370 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464005430 con 0x7fa4680b3370 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa468157090 con 0x7fa4680b3370 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4640038c0 con 0x7fa4680b3370 2023-11-24T06:24:29.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.446+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4680cd4f0 con 0x7fa4680b3370 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.447+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa468067f60 con 0x7fa4680b3370 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.448+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa464003a20 con 0x7fa4680b3370 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.448+0000 7fa45e7fc700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 0x7fa44c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.448+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa464088940 con 0x7fa4680b3370 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.448+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 0x7fa44c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.449+0000 7fa46d6f5700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 0x7fa44c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa458009c40 tx=0x7fa458008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.453+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4640539e0 con 0x7fa4680b3370 2023-11-24T06:24:29.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.644+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4680b7c60 con 0x7fa4680b3370 2023-11-24T06:24:29.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.645+0000 7fa45e7fc700 1 -- 172.21.15.114:0/3213271364 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fa464057000 con 0x7fa4680b3370 2023-11-24T06:24:29.649 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:29.649 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 msgr2=0x7fa44c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 0x7fa44c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa458009c40 tx=0x7fa458008030 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 msgr2=0x7fa4680ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c450 tx=0x7fa46400c810 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 shutdown_connections 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa44c06aed0 0x7fa44c06d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa46815bdd0 0x7fa4680ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4680b3370 0x7fa4680ce2b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 --2- 172.21.15.114:0/3213271364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4680d1c70 0x7fa468156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 >> 172.21.15.114:0/3213271364 conn(0x7fa4680731a0 msgr2=0x7fa4680aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 shutdown_connections 2023-11-24T06:24:29.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.647+0000 7fa47015a700 1 -- 172.21.15.114:0/3213271364 wait complete. 2023-11-24T06:24:29.663 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:24:29.663 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:29.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.781+0000 7fa91a211700 1 Processor -- start 2023-11-24T06:24:29.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.781+0000 7fa91a211700 1 -- start start 2023-11-24T06:24:29.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa91a211700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa914152cc0 0x7fa91406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa91a211700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9140d0630 0x7fa91406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa91a211700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91415c4b0 con 0x7fa9140b49a0 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa91a211700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415c790 con 0x7fa914152cc0 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa91a211700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415ca70 con 0x7fa9140d0630 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa914152cc0 0x7fa91406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa914152cc0 0x7fa91406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37904/0 (socket says 172.21.15.114:37904) 2023-11-24T06:24:29.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 -- 172.21.15.114:0/3052481274 learned_addr learned my addr 172.21.15.114:0/3052481274 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa913fff700 1 --2- 172.21.15.114:0/3052481274 >> v2:172.21.15.172:3300/0 conn(0x7fa9140d0630 0x7fa91406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 -- 172.21.15.114:0/3052481274 >> v2:172.21.15.172:3300/0 conn(0x7fa9140d0630 msgr2=0x7fa91406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 --2- 172.21.15.114:0/3052481274 >> v2:172.21.15.172:3300/0 conn(0x7fa9140d0630 0x7fa91406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 -- 172.21.15.114:0/3052481274 >> v1:172.21.15.172:6789/0 conn(0x7fa9140b49a0 legacy=0x7fa9140cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.782+0000 7fa9137fe700 1 -- 172.21.15.114:0/3052481274 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91415cd50 con 0x7fa914152cc0 2023-11-24T06:24:29.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa9137fe700 1 --2- 172.21.15.114:0/3052481274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa91406c000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b4e80 tx=0x7fa8fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ce3aa8dfa331f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa9127fc700 1 -- 172.21.15.114:0/3052481274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011830 con 0x7fa914152cc0 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa9127fc700 1 -- 172.21.15.114:0/3052481274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc011990 con 0x7fa914152cc0 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa9127fc700 1 -- 172.21.15.114:0/3052481274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011d20 con 0x7fa914152cc0 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 -- 172.21.15.114:0/3052481274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 msgr2=0x7fa91406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 --2- 172.21.15.114:0/3052481274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa91406c000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b4e80 tx=0x7fa8fc009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 -- 172.21.15.114:0/3052481274 shutdown_connections 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 --2- 172.21.15.114:0/3052481274 >> v2:172.21.15.172:3300/0 conn(0x7fa9140d0630 0x7fa91406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 --2- 172.21.15.114:0/3052481274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa91406c000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 -- 172.21.15.114:0/3052481274 >> 172.21.15.114:0/3052481274 conn(0x7fa914071b90 msgr2=0x7fa914071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 -- 172.21.15.114:0/3052481274 shutdown_connections 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.783+0000 7fa91a211700 1 -- 172.21.15.114:0/3052481274 wait complete. 2023-11-24T06:24:29.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.784+0000 7fa91a211700 1 Processor -- start 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 -- start start 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9140b49a0 0x7fa91414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9140d0630 0x7fa914155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9141532b0 con 0x7fa914152cc0 2023-11-24T06:24:29.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914153590 con 0x7fa9140b49a0 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa91a211700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa914153870 con 0x7fa9140d0630 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa913fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa913fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37914/0 (socket says 172.21.15.114:37914) 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.785+0000 7fa913fff700 1 -- 172.21.15.114:0/2102434342 learned_addr learned my addr 172.21.15.114:0/2102434342 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa9137fe700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9140b49a0 0x7fa91414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa912ffd700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9140d0630 0x7fa914155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 -- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9140d0630 msgr2=0x7fa914155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9140d0630 0x7fa914155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 -- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9140b49a0 msgr2=0x7fa91414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9140b49a0 0x7fa91414fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 -- 172.21.15.114:0/2102434342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009550 con 0x7fa914152cc0 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa913fff700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00efc0 tx=0x7fa90c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cd40 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00f040 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0146c0 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa914153b50 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.786+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa914154110 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.787+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9141542d0 con 0x7fa914152cc0 2023-11-24T06:24:29.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.787+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa90c014820 con 0x7fa914152cc0 2023-11-24T06:24:29.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.788+0000 7fa90bfff700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 0x7fa8f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:29.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.788+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa90c089e80 con 0x7fa914152cc0 2023-11-24T06:24:29.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.788+0000 7fa9137fe700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 0x7fa8f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:29.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.788+0000 7fa9137fe700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 0x7fa8f406d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc014040 tx=0x7fa8fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:29.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.792+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa90c054fc0 con 0x7fa914152cc0 2023-11-24T06:24:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.985+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9140753f0 con 0x7fa914152cc0 2023-11-24T06:24:29.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.986+0000 7fa90bfff700 1 -- 172.21.15.114:0/2102434342 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fa90c018090 con 0x7fa914152cc0 2023-11-24T06:24:29.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:29.991 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 msgr2=0x7fa8f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 0x7fa8f406d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc014040 tx=0x7fa8fc008030 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 msgr2=0x7fa9141556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00efc0 tx=0x7fa90c00c3a0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 shutdown_connections 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa8f406aed0 0x7fa8f406d390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa914152cc0 0x7fa9141556c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9140b49a0 0x7fa91414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 --2- 172.21.15.114:0/2102434342 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9140d0630 0x7fa914155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 >> 172.21.15.114:0/2102434342 conn(0x7fa914071b90 msgr2=0x7fa91415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 shutdown_connections 2023-11-24T06:24:29.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:29.989+0000 7fa91a211700 1 -- 172.21.15.114:0/2102434342 wait complete. 2023-11-24T06:24:30.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:24:30.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:24:30.006 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:24:30.006 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:24:30.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:24:30.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:24:30.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:24:30.008 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:24:30.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.212+0000 7f9479dbc700 1 Processor -- start 2023-11-24T06:24:30.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.212+0000 7f68eb228700 1 Processor -- start 2023-11-24T06:24:30.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.212+0000 7f68eb228700 1 -- start start 2023-11-24T06:24:30.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7efc05e43700 1 Processor -- start 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.212+0000 7f9479dbc700 1 -- start start 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9479dbc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9474162590 0x7f947406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9479dbc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9474073f90 0x7f94740743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9479dbc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94740cff40 con 0x7f94740ce220 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9479dbc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740748f0 con 0x7f9474073f90 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9479dbc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740ce6b0 con 0x7f9474162590 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9473fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9474073f90 0x7f94740743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9473fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9474073f90 0x7f94740743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37922/0 (socket says 172.21.15.114:37922) 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7f9473fff700 1 -- 172.21.15.114:0/800218408 learned_addr learned my addr 172.21.15.114:0/800218408 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f94737fe700 1 --2- 172.21.15.114:0/800218408 >> v2:172.21.15.172:3300/0 conn(0x7f9474162590 0x7f947406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68eb228700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f68e40d0520 0x7f68e415d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68eb228700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e415d970 con 0x7f68e4071be0 2023-11-24T06:24:30.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68eb228700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e40723c0 con 0x7f68e40b3b50 2023-11-24T06:24:30.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68eb228700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e4072d60 con 0x7f68e40d0520 2023-11-24T06:24:30.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68e3fff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f68e40b3b50 0x7f68e40714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47230/0 (socket says 172.21.15.114:47230) 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68e3fff700 1 -- 172.21.15.114:0/2645022160 learned_addr learned my addr 172.21.15.114:0/2645022160 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.213+0000 7efc05e43700 1 -- start start 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efc05e43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc000b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efc05e43700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc000720e0 con 0x7efc000b4090 2023-11-24T06:24:30.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efc05e43700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc000723c0 con 0x7efc00071be0 2023-11-24T06:24:30.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efc05e43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc00072d60 con 0x7efc0015d6a0 2023-11-24T06:24:30.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f68e8fc4700 1 --2- 172.21.15.114:0/2645022160 >> v2:172.21.15.114:3300/0 conn(0x7f68e40d0520 0x7f68e415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efbfffff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7efc00071be0 0x7efc00140fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47238/0 (socket says 172.21.15.114:47238) 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efbfffff700 1 -- 172.21.15.114:0/60914932 learned_addr learned my addr 172.21.15.114:0/60914932 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7efbff7fe700 1 --2- 172.21.15.114:0/60914932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc000b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbfe7fc700 1 -- 172.21.15.114:0/60914932 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354723545 0 0) 0x7efc000723c0 con 0x7efc00071be0 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbfe7fc700 1 -- 172.21.15.114:0/60914932 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe80035b0 con 0x7efc00071be0 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbff7fe700 1 -- 172.21.15.114:0/60914932 >> v1:172.21.15.172:6789/0 conn(0x7efc00071be0 legacy=0x7efc00140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbff7fe700 1 -- 172.21.15.114:0/60914932 >> v1:172.21.15.114:6789/0 conn(0x7efc000b4090 legacy=0x7efc000714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbff7fe700 1 -- 172.21.15.114:0/60914932 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc000726a0 con 0x7efc0015d6a0 2023-11-24T06:24:30.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f68e8fc4700 1 -- 172.21.15.114:0/2645022160 >> v1:172.21.15.172:6789/0 conn(0x7f68e40b3b50 legacy=0x7f68e40714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f68e8fc4700 1 -- 172.21.15.114:0/2645022160 >> v1:172.21.15.114:6789/0 conn(0x7f68e4071be0 legacy=0x7f68e4140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f68e8fc4700 1 -- 172.21.15.114:0/2645022160 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4160ba0 con 0x7f68e40d0520 2023-11-24T06:24:30.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbff7fe700 1 --2- 172.21.15.114:0/60914932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc000b3b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efbf00091e0 tx=0x7efbf00094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8109ba804741b8e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f68e8fc4700 1 --2- 172.21.15.114:0/2645022160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e415d430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097c0 tx=0x7f68d4009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d62855f28d5983 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbfe7fc700 1 -- 172.21.15.114:0/60914932 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000c880 con 0x7efc0015d6a0 2023-11-24T06:24:30.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbfe7fc700 1 -- 172.21.15.114:0/60914932 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf000c9e0 con 0x7efc0015d6a0 2023-11-24T06:24:30.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7efbfe7fc700 1 -- 172.21.15.114:0/60914932 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000cd50 con 0x7efc0015d6a0 2023-11-24T06:24:30.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f68e37fe700 1 -- 172.21.15.114:0/2645022160 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4004230 con 0x7f68e40d0520 2023-11-24T06:24:30.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f68e37fe700 1 -- 172.21.15.114:0/2645022160 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4004390 con 0x7f68e40d0520 2023-11-24T06:24:30.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f68e37fe700 1 -- 172.21.15.114:0/2645022160 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40036a0 con 0x7f68e40d0520 2023-11-24T06:24:30.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7efc05e43700 1 -- 172.21.15.114:0/60914932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 msgr2=0x7efc000b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7efc05e43700 1 --2- 172.21.15.114:0/60914932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc000b3b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efbf00091e0 tx=0x7efbf00094f0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7fbb53d5f700 1 Processor -- start 2023-11-24T06:24:30.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7fbb53d5f700 1 -- start start 2023-11-24T06:24:30.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7efc05e43700 1 -- 172.21.15.114:0/60914932 shutdown_connections 2023-11-24T06:24:30.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7efc05e43700 1 --2- 172.21.15.114:0/60914932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc000b3b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7efc05e43700 1 -- 172.21.15.114:0/60914932 >> 172.21.15.114:0/60914932 conn(0x7efc000cfa30 msgr2=0x7efc000cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f9473fff700 1 -- 172.21.15.114:0/800218408 >> v2:172.21.15.172:3300/0 conn(0x7f9474162590 msgr2=0x7f947406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f9473fff700 1 --2- 172.21.15.114:0/800218408 >> v2:172.21.15.172:3300/0 conn(0x7f9474162590 0x7f947406ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f9473fff700 1 -- 172.21.15.114:0/800218408 >> v1:172.21.15.172:6789/0 conn(0x7f94740ce220 legacy=0x7f94740d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.214+0000 7f9473fff700 1 -- 172.21.15.114:0/800218408 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947406c100 con 0x7f9474073f90 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f9473fff700 1 --2- 172.21.15.114:0/800218408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474073f90 0x7f94740743b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9474068fd0 tx=0x7f9468009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39f07098c954ece server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f94727fc700 1 -- 172.21.15.114:0/800218408 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004230 con 0x7f9474073f90 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f94727fc700 1 -- 172.21.15.114:0/800218408 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468004390 con 0x7f9474073f90 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.215+0000 7f94727fc700 1 -- 172.21.15.114:0/800218408 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680036a0 con 0x7f9474073f90 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 -- 172.21.15.114:0/800218408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474073f90 msgr2=0x7f94740743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 --2- 172.21.15.114:0/800218408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474073f90 0x7f94740743b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9474068fd0 tx=0x7f9468009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 -- 172.21.15.114:0/800218408 shutdown_connections 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 --2- 172.21.15.114:0/800218408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474073f90 0x7f94740743b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 --2- 172.21.15.114:0/800218408 >> v2:172.21.15.172:3300/0 conn(0x7f9474162590 0x7f947406ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.216+0000 7f9479dbc700 1 -- 172.21.15.114:0/800218408 >> 172.21.15.114:0/800218408 conn(0x7f94740cd570 msgr2=0x7f94740cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f9479dbc700 1 -- 172.21.15.114:0/800218408 shutdown_connections 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 -- 172.21.15.114:0/2645022160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 msgr2=0x7f68e415d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 --2- 172.21.15.114:0/2645022160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e415d430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097c0 tx=0x7f68d4009ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7efc05e43700 1 -- 172.21.15.114:0/60914932 shutdown_connections 2023-11-24T06:24:30.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 -- 172.21.15.114:0/2645022160 shutdown_connections 2023-11-24T06:24:30.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 --2- 172.21.15.114:0/2645022160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e415d430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 -- 172.21.15.114:0/2645022160 >> 172.21.15.114:0/2645022160 conn(0x7f68e40cfa30 msgr2=0x7f68e40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 -- 172.21.15.114:0/2645022160 shutdown_connections 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 0x7fbb4c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c06d4d0 0x7fbb4c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb4c163a80 0x7fbb4c163f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c06ced0 con 0x7fbb4c163a80 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c06da70 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb53d5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0b4470 con 0x7fbb4c06d4d0 2023-11-24T06:24:30.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb522fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb4c163a80 0x7fbb4c163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f68eb228700 1 -- 172.21.15.114:0/2645022160 wait complete. 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb522fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb4c163a80 0x7fbb4c163f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37938/0 (socket says 172.21.15.114:37938) 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb522fc700 1 -- 172.21.15.114:0/589572501 learned_addr learned my addr 172.21.15.114:0/589572501 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7fbb51afb700 1 --2- 172.21.15.114:0/589572501 >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 0x7fbb4c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 -- 172.21.15.114:0/589572501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c06d4d0 msgr2=0x7fbb4c06eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 --2- 172.21.15.114:0/589572501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c06d4d0 0x7fbb4c06eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 -- 172.21.15.114:0/589572501 >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 msgr2=0x7fbb4c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 --2- 172.21.15.114:0/589572501 >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 0x7fbb4c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 -- 172.21.15.114:0/589572501 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c1634e0 con 0x7fbb4c163a80 2023-11-24T06:24:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb51afb700 1 --2- 172.21.15.114:0/589572501 >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 0x7fbb4c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7fbb522fc700 1 --2- 172.21.15.114:0/589572501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 0x7fbb4c163f10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0750c0 tx=0x7fbb4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc597821fa599257 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7f68eb228700 1 Processor -- start 2023-11-24T06:24:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.217+0000 7f9479dbc700 1 -- 172.21.15.114:0/800218408 wait complete. 2023-11-24T06:24:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7f9479dbc700 1 Processor -- start 2023-11-24T06:24:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.218+0000 7f6b0593a700 1 Processor -- start 2023-11-24T06:24:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7f6b0593a700 1 -- start start 2023-11-24T06:24:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7fbb50af9700 1 -- 172.21.15.114:0/589572501 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400e070 con 0x7fbb4c163a80 2023-11-24T06:24:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7fbb50af9700 1 -- 172.21.15.114:0/589572501 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4400b9c0 con 0x7fbb4c163a80 2023-11-24T06:24:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7fbb50af9700 1 -- 172.21.15.114:0/589572501 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400bd70 con 0x7fbb4c163a80 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7f68eb228700 1 -- start start 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68e4071be0 0x7f68e41408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 0x7f68e40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e416a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40cdd60 con 0x7f68e40d0520 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4167f90 con 0x7f68e40b3b50 2023-11-24T06:24:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68eb228700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4140690 con 0x7f68e4071be0 2023-11-24T06:24:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.219+0000 7f9479dbc700 1 -- start start 2023-11-24T06:24:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 0x7f68e40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9474073f90 0x7f9474144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 0x7f9474142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474145010 0x7f9474146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741433c0 con 0x7f9474145010 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741665f0 con 0x7f9474073f90 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9479dbc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94741668d0 con 0x7f94740ce220 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 0x7f68e40cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33600/0 (socket says 172.21.15.114:33600) 2023-11-24T06:24:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f68e3fff700 1 -- 172.21.15.114:0/2331437273 learned_addr learned my addr 172.21.15.114:0/2331437273 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9472ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 0x7f9474142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9472ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 0x7f9474142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60726/0 (socket says 172.21.15.114:60726) 2023-11-24T06:24:30.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f9472ffd700 1 -- 172.21.15.114:0/483634684 learned_addr learned my addr 172.21.15.114:0/483634684 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7efc05e43700 1 -- 172.21.15.114:0/60914932 wait complete. 2023-11-24T06:24:30.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9473fff700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474145010 0x7f9474146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f94737fe700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9474073f90 0x7f9474144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 -- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9474073f90 msgr2=0x7f9474144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9474073f90 0x7f9474144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 -- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474145010 msgr2=0x7f9474146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9474145010 0x7f9474146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009550 con 0x7f94740ce220 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6b0593a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 0x7f6b000d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6b0593a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 0x7f6b000d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6b0593a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000d2d10 con 0x7f6b0015dfc0 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6b0593a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b000cefd0 con 0x7f6b000d1570 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6b0593a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000cf2b0 con 0x7f6b000b2fc0 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6afeffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6b0015dfc0 0x7f6b000d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53652/0 (socket says 172.21.15.114:53652) 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6afeffd700 1 -- 172.21.15.114:0/1650831750 learned_addr learned my addr 172.21.15.114:0/1650831750 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/1650831750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 0x7f6b000d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/1650831750 >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 0x7f6b000d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6aff7fe700 1 -- 172.21.15.114:0/1650831750 >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 msgr2=0x7f6b000d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/1650831750 >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 0x7f6b000d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6aff7fe700 1 -- 172.21.15.114:0/1650831750 >> v1:172.21.15.114:6789/0 conn(0x7f6b0015dfc0 legacy=0x7f6b000d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6afdffb700 1 -- 172.21.15.114:0/1650831750 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125539115 0 0) 0x7f6b000d2d10 con 0x7f6b0015dfc0 2023-11-24T06:24:30.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 -- 172.21.15.114:0/589572501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 msgr2=0x7fbb4c163f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/589572501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 0x7fbb4c163f10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0750c0 tx=0x7fbb4400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e97c5700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e416a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6aff7fe700 1 -- 172.21.15.114:0/1650831750 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000d2fb0 con 0x7f6b000b2fc0 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/1650831750 >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 0x7f6b000d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 -- 172.21.15.114:0/589572501 shutdown_connections 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/589572501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 0x7fbb4c163f10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/589572501 >> v2:172.21.15.172:3300/0 conn(0x7fbb4c0b3b50 0x7fbb4c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/589572501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c06d4d0 0x7fbb4c06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7fbb53d5f700 1 -- 172.21.15.114:0/589572501 >> 172.21.15.114:0/589572501 conn(0x7fbb4c0cfa30 msgr2=0x7fbb4c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 -- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68e4071be0 msgr2=0x7f68e41408a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68e4071be0 0x7f68e41408a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 -- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 msgr2=0x7f68e416a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68e40d0520 0x7f68e416a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 -- 172.21.15.114:0/2331437273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d40094a0 con 0x7f68e40b3b50 2023-11-24T06:24:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f9472ffd700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 0x7f9474142e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f946400b470 tx=0x7f946400b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f68e3fff700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 0x7f68e40cce30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68dc002020 tx=0x7f68dc00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fbb53d5f700 1 -- 172.21.15.114:0/589572501 shutdown_connections 2023-11-24T06:24:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7fc34a72a700 1 Processor -- start 2023-11-24T06:24:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.220+0000 7fc34a72a700 1 -- start start 2023-11-24T06:24:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc34a72a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc34a72a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc34406d4d0 0x7fc34406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc34a72a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34406ced0 con 0x7fc344163a80 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc34a72a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34406da70 con 0x7fc34406d4d0 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc34a72a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440b4470 con 0x7fc3440b3b50 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc343fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc3437fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc34406d4d0 0x7fc34406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc3437fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc34406d4d0 0x7fc34406eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37958/0 (socket says 172.21.15.114:37958) 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fc3437fe700 1 -- 172.21.15.114:0/3584381201 learned_addr learned my addr 172.21.15.114:0/3584381201 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7efc05e43700 1 Processor -- start 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011830 con 0x7f94740ce220 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640119b0 con 0x7f94740ce220 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011d60 con 0x7f94740ce220 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9474166bb0 con 0x7f94740ce220 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9474143ad0 con 0x7f94740ce220 2023-11-24T06:24:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7fbb53d5f700 1 -- 172.21.15.114:0/589572501 wait complete. 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00ad60 con 0x7f68e40b3b50 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc0070d0 con 0x7f68e40b3b50 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc007480 con 0x7f68e40b3b50 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e40cd4c0 con 0x7f68e40b3b50 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e40cda80 con 0x7f68e40b3b50 2023-11-24T06:24:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7fc3437fe700 1 -- 172.21.15.114:0/3584381201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 msgr2=0x7fc3440b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7fc3437fe700 1 --2- 172.21.15.114:0/3584381201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7fc3437fe700 1 -- 172.21.15.114:0/3584381201 >> v1:172.21.15.114:6789/0 conn(0x7fc344163a80 legacy=0x7fc344163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7fc3437fe700 1 -- 172.21.15.114:0/3584381201 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3441634e0 con 0x7fc34406d4d0 2023-11-24T06:24:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9474069070 con 0x7f94740ce220 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f25d8e57700 1 Processor -- start 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.221+0000 7f25d8e57700 1 -- start start 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f25d8e57700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 0x7f25d406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7f25d8e57700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25d4072bb0 0x7f25d40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7f25d8e57700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 0x7f25d416e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7f25d8e57700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d40b4510 con 0x7f25d40b4090 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7f25d8e57700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d408bdb0 con 0x7f25d4072bb0 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7f25d8e57700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d406ce30 con 0x7f25d416ae60 2023-11-24T06:24:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d2d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 0x7f25d416e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7fc3437fe700 1 --2- 172.21.15.114:0/3584381201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc34406d4d0 0x7fc34406eee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098f0 tx=0x7fc338009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fba6072eb3c129f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d2d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 0x7f25d416e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37968/0 (socket says 172.21.15.114:37968) 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.223+0000 7fbb53d5f700 1 Processor -- start 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7fbb53d5f700 1 -- start start 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7efc05e43700 1 -- start start 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 0x7efc00152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc000b4090 0x7efc001f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc001558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc001534f0 con 0x7efc00071be0 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00140410 con 0x7efc000b4090 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efc05e43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc001f8e90 con 0x7efc0015d6a0 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7fc342ffd700 1 -- 172.21.15.114:0/3584381201 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338004230 con 0x7fc34406d4d0 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 0x7efc00152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 0x7efc00152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37970/0 (socket says 172.21.15.114:37970) 2023-11-24T06:24:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 -- 172.21.15.114:0/1086675924 learned_addr learned my addr 172.21.15.114:0/1086675924 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbfffff700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc001558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d2d9d700 1 -- 172.21.15.114:0/11529597 learned_addr learned my addr 172.21.15.114:0/11529597 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 --2- 172.21.15.114:0/11529597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 0x7f25d406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d1d9b700 1 --2- 172.21.15.114:0/11529597 >> v2:172.21.15.172:3300/0 conn(0x7f25d4072bb0 0x7f25d40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7fc342ffd700 1 -- 172.21.15.114:0/3584381201 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338004390 con 0x7fc34406d4d0 2023-11-24T06:24:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7fc342ffd700 1 -- 172.21.15.114:0/3584381201 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380036a0 con 0x7fc34406d4d0 2023-11-24T06:24:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 -- 172.21.15.114:0/11529597 >> v2:172.21.15.172:3300/0 conn(0x7f25d4072bb0 msgr2=0x7f25d40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 --2- 172.21.15.114:0/11529597 >> v2:172.21.15.172:3300/0 conn(0x7f25d4072bb0 0x7f25d40b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 -- 172.21.15.114:0/11529597 >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 msgr2=0x7f25d416e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 --2- 172.21.15.114:0/11529597 >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 0x7f25d416e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7f25d259c700 1 -- 172.21.15.114:0/11529597 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d406d110 con 0x7f25d416ae60 2023-11-24T06:24:30.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 -- 172.21.15.114:0/1086675924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 msgr2=0x7efc001558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc0015d6a0 0x7efc001558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 -- 172.21.15.114:0/1086675924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc000b4090 msgr2=0x7efc001f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc000b4090 0x7efc001f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.224+0000 7efbff7fe700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf0009d30 con 0x7efc00071be0 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7efbff7fe700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 0x7efc00152fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efbf0000c00 tx=0x7efbf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0003ab0 con 0x7efc00071be0 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0003c50 con 0x7efc00071be0 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00045d0 con 0x7efc00071be0 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fc34a72a700 1 -- 172.21.15.114:0/3584381201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc34406d4d0 msgr2=0x7fc34406eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fc34a72a700 1 --2- 172.21.15.114:0/3584381201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc34406d4d0 0x7fc34406eee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098f0 tx=0x7fc338009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb4c06d4d0 0x7fbb4c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 0x7fbb4c1eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 0x7fbb4c1eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1574f0 con 0x7fbb4c163a80 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1ec6e0 con 0x7fbb4c06d4d0 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb53d5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1eb640 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb512fa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 0x7fbb4c1eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb512fa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 0x7fbb4c1eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60752/0 (socket says 172.21.15.114:60752) 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb512fa700 1 -- 172.21.15.114:0/2237258384 learned_addr learned my addr 172.21.15.114:0/2237258384 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7fbb51afb700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb4c06d4d0 0x7fbb4c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 -- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb4c06d4d0 msgr2=0x7fbb4c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb4c06d4d0 0x7fbb4c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 -- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 msgr2=0x7fbb4c1eafc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb4c163a80 0x7fbb4c1eafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009b30 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.222+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/1650831750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 0x7f6b000d3a10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009d90 tx=0x7f6af800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af5152054a86c42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7f6afdffb700 1 -- 172.21.15.114:0/1650831750 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800e070 con 0x7f6b000b2fc0 2023-11-24T06:24:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7f6afdffb700 1 -- 172.21.15.114:0/1650831750 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af800bbd0 con 0x7f6b000b2fc0 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.225+0000 7f6afdffb700 1 -- 172.21.15.114:0/1650831750 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8012430 con 0x7f6b000b2fc0 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 -- 172.21.15.114:0/1650831750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 msgr2=0x7f6b000d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 --2- 172.21.15.114:0/1650831750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 0x7f6b000d3a10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009d90 tx=0x7f6af800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 -- 172.21.15.114:0/1650831750 shutdown_connections 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 --2- 172.21.15.114:0/1650831750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000b2fc0 0x7f6b000d3a10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 --2- 172.21.15.114:0/1650831750 >> v2:172.21.15.172:3300/0 conn(0x7f6b000d1570 0x7f6b000d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 -- 172.21.15.114:0/1650831750 >> 172.21.15.114:0/1650831750 conn(0x7f6b0006b410 msgr2=0x7f6b0006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f6b0593a700 1 -- 172.21.15.114:0/1650831750 shutdown_connections 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 -- 172.21.15.114:0/3584381201 shutdown_connections 2023-11-24T06:24:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 --2- 172.21.15.114:0/3584381201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 --2- 172.21.15.114:0/3584381201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc34406d4d0 0x7fc34406eee0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 -- 172.21.15.114:0/3584381201 >> 172.21.15.114:0/3584381201 conn(0x7fc3440cfa30 msgr2=0x7fc3440cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb512fa700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 0x7fbb4c1eaa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800b470 tx=0x7fbb4800b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 -- 172.21.15.114:0/3584381201 shutdown_connections 2023-11-24T06:24:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48011830 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48011990 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48011d40 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f946401a400 con 0x7f94740ce220 2023-11-24T06:24:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c1eb920 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1ebee0 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7fc34a72a700 1 -- 172.21.15.114:0/3584381201 wait complete. 2023-11-24T06:24:30.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68dc0075e0 con 0x7f68e40b3b50 2023-11-24T06:24:30.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f68e17fa700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f68d006afa0 0x7f68d006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68dc089650 con 0x7f68e40b3b50 2023-11-24T06:24:30.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc001565d0 con 0x7efc00071be0 2023-11-24T06:24:30.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.226+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc00156b90 con 0x7efc00071be0 2023-11-24T06:24:30.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f25d259c700 1 --2- 172.21.15.114:0/11529597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 0x7f25d406efc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25d4074da0 tx=0x7f25cc00f650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3da7a7b29d81469 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc001416e0 con 0x7efc00071be0 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f945bfff700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f946006af20 0x7f946006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f94737fe700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f946006af20 0x7f946006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9464089b20 con 0x7f94740ce220 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f945bfff700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f9460073140 0x7f9460075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9460075c10 con 0x7f9460073140 2023-11-24T06:24:30.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f68e8fc4700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f68d006afa0 0x7f68d006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f9473fff700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f9460073140 0x7f9460075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb4c06e2c0 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f68eb228700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7f68c40015d0 0x7f68c4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 --> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68c4006bc0 con 0x7f68c40015d0 2023-11-24T06:24:30.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9464089fc0 con 0x7f94740ce220 2023-11-24T06:24:30.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f25d159a700 1 -- 172.21.15.114:0/11529597 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc012070 con 0x7f25d416ae60 2023-11-24T06:24:30.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.227+0000 7f25d159a700 1 -- 172.21.15.114:0/11529597 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc00fe40 con 0x7f25d416ae60 2023-11-24T06:24:30.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f25d159a700 1 -- 172.21.15.114:0/11529597 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00d310 con 0x7f25d416ae60 2023-11-24T06:24:30.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f68e8fc4700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f68d006afa0 0x7f68d006d460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009b10 tx=0x7f68d4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7fc34a72a700 1 Processor -- start 2023-11-24T06:24:30.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f94737fe700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f946006af20 0x7f946006d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f947406cf50 tx=0x7f946c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f9473fff700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f9460073140 0x7f9460075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f68e97c5700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7f68c40015d0 0x7f68c4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.228+0000 7f6b0593a700 1 -- 172.21.15.114:0/1650831750 wait complete. 2023-11-24T06:24:30.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== osd.3 v2:172.21.15.114:6816/783795072 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9460075c10 con 0x7f9460073140 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f6b0593a700 1 Processor -- start 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efbf0022450 con 0x7efc00071be0 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbec06aed0 0x7efbec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbf008b400 con 0x7efc00071be0 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7efbec0730f0 0x7efbec075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbec075bc0 con 0x7efbec0730f0 2023-11-24T06:24:30.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbf008b7b0 con 0x7efc00071be0 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 -- 172.21.15.114:0/11529597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 msgr2=0x7f25d406efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 --2- 172.21.15.114:0/11529597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 0x7f25d406efc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25d4074da0 tx=0x7f25cc00f650 comp rx=0 tx=0).stop 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 -- 172.21.15.114:0/11529597 shutdown_connections 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 --2- 172.21.15.114:0/11529597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d416ae60 0x7f25d406efc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 --2- 172.21.15.114:0/11529597 >> v2:172.21.15.114:3300/0 conn(0x7f25d40b4090 0x7f25d416e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 --2- 172.21.15.114:0/11529597 >> v2:172.21.15.172:3300/0 conn(0x7f25d4072bb0 0x7f25d40b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 -- 172.21.15.114:0/11529597 >> 172.21.15.114:0/11529597 conn(0x7f25d40cfa30 msgr2=0x7f25d40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 -- 172.21.15.114:0/11529597 shutdown_connections 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7f25d8e57700 1 -- 172.21.15.114:0/11529597 wait complete. 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb4801a400 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7fbb427fc700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbb3006aed0 0x7fbb3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb480895f0 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fbb427fc700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7fbb30073070 0x7fbb30075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb30075b40 con 0x7fbb30073070 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4808bc50 con 0x7fbb4c0b3b50 2023-11-24T06:24:30.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fbb522fc700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7fbb30073070 0x7fbb30075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7f6b0593a700 1 -- start start 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b000b2fc0 0x7f6b000c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000d1570 0x7f6b000cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b0015dfc0 0x7f6b000c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000c93d0 con 0x7f6b0015dfc0 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000cc100 con 0x7f6b000b2fc0 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6b0593a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000c8d80 con 0x7f6b000d1570 2023-11-24T06:24:30.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b000b2fc0 0x7f6b000c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b000b2fc0 0x7f6b000c5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33658/0 (socket says 172.21.15.114:33658) 2023-11-24T06:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6afeffd700 1 -- 172.21.15.114:0/3372653004 learned_addr learned my addr 172.21.15.114:0/3372653004 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000d1570 0x7f6b000cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7efbfeffd700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbec06aed0 0x7efbec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f25d8e57700 1 Processor -- start 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fbb51afb700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbb3006aed0 0x7fbb3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.229+0000 7fc34a72a700 1 -- start start 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34406d4d0 0x7fc34415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc344163a80 0x7fc34413ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440ce620 con 0x7fc344163a80 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34413dbb0 con 0x7fc34406d4d0 2023-11-24T06:24:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc34a72a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415d4c0 con 0x7fc3440b3b50 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.230+0000 7fc3437fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60764/0 (socket says 172.21.15.114:60764) 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 -- 172.21.15.114:0/2018133439 learned_addr learned my addr 172.21.15.114:0/2018133439 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 -- 172.21.15.114:0/2018133439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34406d4d0 msgr2=0x7fc34415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34406d4d0 0x7fc34415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 -- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc344163a80 msgr2=0x7fc34413ae30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc344163a80 0x7fc34413ae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7fc3437fe700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009550 con 0x7fc3440b3b50 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7efbfffff700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7efbec0730f0 0x7efbec075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.231+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b0015dfc0 0x7f6b000c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 -- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b000b2fc0 msgr2=0x7f6b000c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b000b2fc0 0x7f6b000c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 -- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b0015dfc0 msgr2=0x7f6b000c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b0015dfc0 0x7f6b000c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7fbb522fc700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7fbb30073070 0x7fbb30075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8009a70 con 0x7f6b000d1570 2023-11-24T06:24:30.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f68e97c5700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7f68c40015d0 0x7f68c4003a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7fbb51afb700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbb3006aed0 0x7fbb3006d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c900 tx=0x7fbb3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7efbfeffd700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbec06aed0 0x7efbec06d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000f60 tx=0x7efbf4007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6afe7fc700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000d1570 0x7f6b000cb800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e810 tx=0x7f6af000ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7efbfffff700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7efbec0730f0 0x7efbec075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004230 con 0x7f6b000d1570 2023-11-24T06:24:30.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0004390 con 0x7f6b000d1570 2023-11-24T06:24:30.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00036a0 con 0x7f6b000d1570 2023-11-24T06:24:30.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b000c9060 con 0x7f6b000d1570 2023-11-24T06:24:30.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b000cc3e0 con 0x7f6b000d1570 2023-11-24T06:24:30.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== osd.7 v2:172.21.15.172:6808/2370047192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f68c4006bc0 con 0x7f68c40015d0 2023-11-24T06:24:30.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== osd.5 v2:172.21.15.172:6816/1185079648 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbb30075b40 con 0x7fbb30073070 2023-11-24T06:24:30.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b00152e30 con 0x7f6b000d1570 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.232+0000 7f25d8e57700 1 -- start start 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 0x7f25d4163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d40acde0 0x7f25d40aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25d40ab1a0 0x7f25d4203e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40aba20 con 0x7f25d40b4090 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40ab650 con 0x7f25d40ab1a0 2023-11-24T06:24:30.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d8e57700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41677c0 con 0x7f25d40acde0 2023-11-24T06:24:30.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 0x7f25d4163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== osd.1 v2:172.21.15.114:6800/2279805148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7efbec075bc0 con 0x7efbec0730f0 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 0x7f25d4163030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38004/0 (socket says 172.21.15.114:38004) 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d259c700 1 -- 172.21.15.114:0/595903902 learned_addr learned my addr 172.21.15.114:0/595903902 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d2d9d700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25d40ab1a0 0x7f25d4203e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.233+0000 7f25d1d9b700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d40acde0 0x7f25d40aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 -- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d40acde0 msgr2=0x7f25d40aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25d40acde0 0x7f25d40aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 -- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25d40ab1a0 msgr2=0x7f25d4203e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25d40ab1a0 0x7f25d4203e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc00f050 con 0x7f25d40b4090 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d259c700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 0x7f25d4163030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25d406cf70 tx=0x7f25cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00cdd0 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc003a20 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d40af390 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc003db0 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.234+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d4167100 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.235+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25d406d9b0 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.236+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25cc004050 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25c2ffd700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25b406aed0 0x7f25b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25cc08ec40 con 0x7f25d40b4090 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25c2ffd700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f25b4073070 0x7f25b4075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25b4075b40 con 0x7f25b4073070 2023-11-24T06:24:30.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25cc08efb0 con 0x7f25d40b4090 2023-11-24T06:24:30.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25d1d9b700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25b406aed0 0x7f25b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25d2d9d700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f25b4073070 0x7f25b4075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25d1d9b700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25b406aed0 0x7f25b406d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25d40acb00 tx=0x7f25c4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.237+0000 7f25d2d9d700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f25b4073070 0x7f25b4075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.238+0000 7fc3437fe700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440ceab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc338000c00 tx=0x7fc338004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.238+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338003c40 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.238+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338020070 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.238+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338011630 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.239+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34413d8d0 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.239+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3440ce2b0 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.240+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc344165bf0 con 0x7fc3440b3b50 2023-11-24T06:24:30.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.240+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc338022030 con 0x7fc3440b3b50 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.240+0000 7fc340ff9700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc33006aed0 0x7fc33006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc33808ae20 con 0x7fc3440b3b50 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc340ff9700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7fc330073070 0x7fc330075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc330075b40 con 0x7fc330073070 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33808d4e0 con 0x7fc3440b3b50 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc343fff700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc33006aed0 0x7fc33006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7fc343fff700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc33006aed0 0x7fc33006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc34413e220 tx=0x7fc334008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6af0003800 con 0x7f6b000d1570 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6aec06aed0 0x7f6aec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6af0089480 con 0x7f6b000d1570 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7f6aec073070 0x7f6aec075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6aec075b40 con 0x7f6aec073070 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.241+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af00898d0 con 0x7f6b000d1570 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.242+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7f6aec073070 0x7f6aec075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.242+0000 7f6aff7fe700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7f6aec073070 0x7f6aec075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.243+0000 7f6afeffd700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6aec06aed0 0x7f6aec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.243+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== osd.4 v2:172.21.15.172:6800/517626883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6aec075b40 con 0x7f6aec073070 2023-11-24T06:24:30.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.243+0000 7f6afeffd700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6aec06aed0 0x7f6aec06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000f60 tx=0x7f6af4007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.250+0000 7fc348cc7700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7fc330073070 0x7fc330075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.239+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== osd.2 v2:172.21.15.114:6817/2005258798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f25b4075b40 con 0x7f25b4073070 2023-11-24T06:24:30.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.250+0000 7fc348cc7700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7fc330073070 0x7fc330075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.253+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== osd.0 v2:172.21.15.114:6808/1249744364 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc330075b40 con 0x7fc330073070 2023-11-24T06:24:30.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.253+0000 7f5193d67700 1 Processor -- start 2023-11-24T06:24:30.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.253+0000 7f5193d67700 1 -- start start 2023-11-24T06:24:30.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5193d67700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518c06d4d0 0x7f518c16cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5193d67700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c06ced0 con 0x7f518c164a90 2023-11-24T06:24:30.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5193d67700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c06da70 con 0x7f518c0b3b50 2023-11-24T06:24:30.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5193d67700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c1634e0 con 0x7f518c06d4d0 2023-11-24T06:24:30.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5191b03700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f518c0b3b50 0x7f518c0b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47326/0 (socket says 172.21.15.114:47326) 2023-11-24T06:24:30.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.255+0000 7f5191b03700 1 -- 172.21.15.114:0/1487484094 learned_addr learned my addr 172.21.15.114:0/1487484094 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.257+0000 7f5192304700 1 --2- 172.21.15.114:0/1487484094 >> v2:172.21.15.172:3300/0 conn(0x7f518c06d4d0 0x7f518c16cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.258+0000 7f5192304700 1 -- 172.21.15.114:0/1487484094 >> v1:172.21.15.172:6789/0 conn(0x7f518c0b3b50 legacy=0x7f518c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.258+0000 7f5192304700 1 -- 172.21.15.114:0/1487484094 >> v1:172.21.15.114:6789/0 conn(0x7f518c164a90 legacy=0x7f518c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.258+0000 7f5192304700 1 -- 172.21.15.114:0/1487484094 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c1637c0 con 0x7f518c06d4d0 2023-11-24T06:24:30.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.258+0000 7f5192304700 1 --2- 172.21.15.114:0/1487484094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c06d4d0 0x7f518c16cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5184009d90 tx=0x7f518400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27206210618b340e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.259+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 --> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94740cfbd0 con 0x7f9460073140 2023-11-24T06:24:30.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.261+0000 7f945bfff700 1 -- 172.21.15.114:0/483634684 <== osd.3 v2:172.21.15.114:6816/783795072 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9468004030 con 0x7f9460073140 2023-11-24T06:24:30.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.261+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f9460073140 msgr2=0x7f9460075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.261+0000 7f9479dbc700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f9460073140 0x7f9460075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.261+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 --> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6b000b93e0 con 0x7f6aec073070 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f946006af20 msgr2=0x7f946006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f946006af20 0x7f946006d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f947406cf50 tx=0x7f946c00b040 comp rx=0 tx=0).stop 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 msgr2=0x7f9474142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 --2- 172.21.15.114:0/483634684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94740ce220 0x7f9474142e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f946400b470 tx=0x7f946400b780 comp rx=0 tx=0).stop 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9473fff700 1 -- 172.21.15.114:0/483634684 reap_dead start 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 shutdown_connections 2023-11-24T06:24:30.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 >> 172.21.15.114:0/483634684 conn(0x7f94740cd570 msgr2=0x7f9474151470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6ae7fff700 1 -- 172.21.15.114:0/3372653004 <== osd.4 v2:172.21.15.172:6800/517626883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6b000b93e0 con 0x7f6aec073070 2023-11-24T06:24:30.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 shutdown_connections 2023-11-24T06:24:30.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f9479dbc700 1 -- 172.21.15.114:0/483634684 wait complete. 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7f6aec073070 msgr2=0x7f6aec075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7f6aec073070 0x7f6aec075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6aec06aed0 msgr2=0x7f6aec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6aec06aed0 0x7f6aec06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6af4000f60 tx=0x7f6af4007ca0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000d1570 msgr2=0x7f6b000cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 --2- 172.21.15.114:0/3372653004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b000d1570 0x7f6b000cb800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e810 tx=0x7f6af000ebd0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6aff7fe700 1 -- 172.21.15.114:0/3372653004 reap_dead start 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 shutdown_connections 2023-11-24T06:24:30.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.262+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 >> 172.21.15.114:0/3372653004 conn(0x7f6b0006b410 msgr2=0x7f6b000b86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.263+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 shutdown_connections 2023-11-24T06:24:30.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.265+0000 7f5190b01700 1 -- 172.21.15.114:0/1487484094 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518400e070 con 0x7f518c06d4d0 2023-11-24T06:24:30.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.265+0000 7f5190b01700 1 -- 172.21.15.114:0/1487484094 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518400bad0 con 0x7f518c06d4d0 2023-11-24T06:24:30.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.265+0000 7f5190b01700 1 -- 172.21.15.114:0/1487484094 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518400be60 con 0x7f518c06d4d0 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 -- 172.21.15.114:0/1487484094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c06d4d0 msgr2=0x7f518c16cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 --2- 172.21.15.114:0/1487484094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c06d4d0 0x7f518c16cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5184009d90 tx=0x7f518400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 -- 172.21.15.114:0/1487484094 shutdown_connections 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 --2- 172.21.15.114:0/1487484094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c06d4d0 0x7f518c16cef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 -- 172.21.15.114:0/1487484094 >> 172.21.15.114:0/1487484094 conn(0x7f518c0cfa30 msgr2=0x7f518c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 -- 172.21.15.114:0/1487484094 shutdown_connections 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f5193d67700 1 -- 172.21.15.114:0/1487484094 wait complete. 2023-11-24T06:24:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 --> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc000751e0 con 0x7efbec0730f0 2023-11-24T06:24:30.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.267+0000 7efbe7fff700 1 -- 172.21.15.114:0/1086675924 <== osd.1 v2:172.21.15.114:6800/2279805148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efc000751e0 con 0x7efbec0730f0 2023-11-24T06:24:30.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.268+0000 7f5193d67700 1 Processor -- start 2023-11-24T06:24:30.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.268+0000 7f5193d67700 1 -- start start 2023-11-24T06:24:30.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.266+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 --> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68c4005ca0 con 0x7f68c40015d0 2023-11-24T06:24:30.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.264+0000 7f6b0593a700 1 -- 172.21.15.114:0/3372653004 wait complete. 2023-11-24T06:24:30.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 --> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25d406d6d0 con 0x7f25b4073070 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c06d4d0 0x7f518c141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 0x7f518c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c164a90 0x7f518c13c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c161260 con 0x7f518c0b3b50 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c071e60 con 0x7f518c164a90 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5193d67700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c160f10 con 0x7f518c06d4d0 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5191302700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 0x7f518c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5191302700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 0x7f518c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38030/0 (socket says 172.21.15.114:38030) 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5191302700 1 -- 172.21.15.114:0/1279141018 learned_addr learned my addr 172.21.15.114:0/1279141018 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f5192304700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c164a90 0x7f518c13c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.269+0000 7f25c2ffd700 1 -- 172.21.15.114:0/595903902 <== osd.2 v2:172.21.15.114:6817/2005258798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25c800be90 con 0x7f25b4073070 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191b03700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c06d4d0 0x7f518c141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 -- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c06d4d0 msgr2=0x7f518c141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c06d4d0 0x7f518c141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 -- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c164a90 msgr2=0x7f518c13c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c164a90 0x7f518c13c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184009a70 con 0x7f518c0b3b50 2023-11-24T06:24:30.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5192304700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c164a90 0x7f518c13c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:30.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f68e17fa700 1 -- 172.21.15.114:0/2331437273 <== osd.7 v2:172.21.15.172:6808/2370047192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68c4005ca0 con 0x7f68c40015d0 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5191302700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 0x7f518c161680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f518800c450 tx=0x7f518800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880054e0 con 0x7f518c0b3b50 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518800ce60 con 0x7f518c0b3b50 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.270+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c15d800 con 0x7f518c0b3b50 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.271+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c15dd80 con 0x7f518c0b3b50 2023-11-24T06:24:30.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.271+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880038c0 con 0x7f518c0b3b50 2023-11-24T06:24:30.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7efbec0730f0 msgr2=0x7efbec075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7efbec0730f0 0x7efbec075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 --> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb4c165bf0 con 0x7fbb30073070 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f25b4073070 msgr2=0x7f25b4075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f25b4073070 0x7f25b4075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25b406aed0 msgr2=0x7f25b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25b406aed0 0x7f25b406d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25d40acb00 tx=0x7f25c4009210 comp rx=0 tx=0).stop 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 msgr2=0x7f25d4163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d8e57700 1 --2- 172.21.15.114:0/595903902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25d40b4090 0x7f25d4163030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25d406cf70 tx=0x7f25cc004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f25d2d9d700 1 -- 172.21.15.114:0/595903902 reap_dead start 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbec06aed0 msgr2=0x7efbec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbec06aed0 0x7efbec06d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000f60 tx=0x7efbf4007a40 comp rx=0 tx=0).stop 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 msgr2=0x7efc00152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7efc05e43700 1 --2- 172.21.15.114:0/1086675924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc00071be0 0x7efc00152fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efbf0000c00 tx=0x7efbf0004750 comp rx=0 tx=0).stop 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7fbb427fc700 1 -- 172.21.15.114:0/2237258384 <== osd.5 v2:172.21.15.172:6816/1185079648 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb44009300 con 0x7fbb30073070 2023-11-24T06:24:30.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7efbfffff700 1 -- 172.21.15.114:0/1086675924 reap_dead start 2023-11-24T06:24:30.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 shutdown_connections 2023-11-24T06:24:30.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 >> 172.21.15.114:0/1086675924 conn(0x7efc000cfa30 msgr2=0x7efc000b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7fbb30073070 msgr2=0x7fbb30075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7fbb30073070 0x7fbb30075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.272+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5188003a20 con 0x7f518c0b3b50 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f518c165bf0 con 0x7f518c0b3b50 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f51827fc700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f517006aed0 0x7f517006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51880888c0 con 0x7f518c0b3b50 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f5191b03700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f517006aed0 0x7f517006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f51827fc700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f5170073070 0x7f5170075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5170075b40 con 0x7f5170073070 2023-11-24T06:24:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518808d2c0 con 0x7f518c0b3b50 2023-11-24T06:24:30.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 shutdown_connections 2023-11-24T06:24:30.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 >> 172.21.15.114:0/595903902 conn(0x7f25d40cfa30 msgr2=0x7f25d416bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f5192304700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f5170073070 0x7f5170075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.273+0000 7f5191b03700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f517006aed0 0x7f517006d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f517c007c20 tx=0x7f517c007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7f68c40015d0 msgr2=0x7f68c4003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7f68c40015d0 0x7f68c4003a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f5192304700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f5170073070 0x7f5170075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f68d006afa0 msgr2=0x7f68d006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f68d006afa0 0x7f68d006d460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009b10 tx=0x7f68d4008030 comp rx=0 tx=0).stop 2023-11-24T06:24:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 msgr2=0x7f68e40cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 --2- 172.21.15.114:0/2331437273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68e40b3b50 0x7f68e40cce30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68dc002020 tx=0x7f68dc00a340 comp rx=0 tx=0).stop 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68e97c5700 1 -- 172.21.15.114:0/2331437273 reap_dead start 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 shutdown_connections 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 >> 172.21.15.114:0/2331437273 conn(0x7f68e40cfa30 msgr2=0x7f68e41436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 shutdown_connections 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7f68eb228700 1 -- 172.21.15.114:0/2331437273 wait complete. 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 shutdown_connections 2023-11-24T06:24:30.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7f25d8e57700 1 -- 172.21.15.114:0/595903902 wait complete. 2023-11-24T06:24:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbb3006aed0 msgr2=0x7fbb3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbb3006aed0 0x7fbb3006d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c900 tx=0x7fbb3c008040 comp rx=0 tx=0).stop 2023-11-24T06:24:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 msgr2=0x7fbb4c1eaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 --2- 172.21.15.114:0/2237258384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb4c0b3b50 0x7fbb4c1eaa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800b470 tx=0x7fbb4800b830 comp rx=0 tx=0).stop 2023-11-24T06:24:30.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== osd.6 v2:172.21.15.172:6821/1378310518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5170075b40 con 0x7f5170073070 2023-11-24T06:24:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb522fc700 1 -- 172.21.15.114:0/2237258384 reap_dead start 2023-11-24T06:24:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 shutdown_connections 2023-11-24T06:24:30.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.275+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 >> 172.21.15.114:0/2237258384 conn(0x7fbb4c0cfa30 msgr2=0x7fbb4c0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.318 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:24:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 shutdown_connections 2023-11-24T06:24:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.274+0000 7efc05e43700 1 -- 172.21.15.114:0/1086675924 wait complete. 2023-11-24T06:24:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.281+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 shutdown_connections 2023-11-24T06:24:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.281+0000 7fbb53d5f700 1 -- 172.21.15.114:0/2237258384 wait complete. 2023-11-24T06:24:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.283+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 --> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc34406cb60 con 0x7fc330073070 2023-11-24T06:24:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.284+0000 7fc340ff9700 1 -- 172.21.15.114:0/2018133439 <== osd.0 v2:172.21.15.114:6808/1249744364 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc33c0018d0 con 0x7fc330073070 2023-11-24T06:24:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.284+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7fc330073070 msgr2=0x7fc330075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.284+0000 7fc34a72a700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7fc330073070 0x7fc330075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc33006aed0 msgr2=0x7fc33006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc33006aed0 0x7fc33006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc34413e220 tx=0x7fc334008040 comp rx=0 tx=0).stop 2023-11-24T06:24:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 msgr2=0x7fc3440ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 --2- 172.21.15.114:0/2018133439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3440b3b50 0x7fc3440ceab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc338000c00 tx=0x7fc338004750 comp rx=0 tx=0).stop 2023-11-24T06:24:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc348cc7700 1 -- 172.21.15.114:0/2018133439 reap_dead start 2023-11-24T06:24:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 shutdown_connections 2023-11-24T06:24:30.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.285+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 >> 172.21.15.114:0/2018133439 conn(0x7fc3440cfa30 msgr2=0x7fc3440b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.290+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 shutdown_connections 2023-11-24T06:24:30.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.291+0000 7fc34a72a700 1 -- 172.21.15.114:0/2018133439 wait complete. 2023-11-24T06:24:30.324 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:24:30.325 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.325 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:24:30.326 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:24:30.327 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:24:30.328 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:30.328 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:24:30.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.302+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 --> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f518c1613a0 con 0x7f5170073070 2023-11-24T06:24:30.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f51827fc700 1 -- 172.21.15.114:0/1279141018 <== osd.6 v2:172.21.15.172:6821/1378310518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f518c1613a0 con 0x7f5170073070 2023-11-24T06:24:30.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f5170073070 msgr2=0x7f5170075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f5170073070 0x7f5170075490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f517006aed0 msgr2=0x7f517006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f517006aed0 0x7f517006d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f517c007c20 tx=0x7f517c007b70 comp rx=0 tx=0).stop 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 msgr2=0x7f518c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 --2- 172.21.15.114:0/1279141018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f518c0b3b50 0x7f518c161680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f518800c450 tx=0x7f518800c760 comp rx=0 tx=0).stop 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5192304700 1 -- 172.21.15.114:0/1279141018 reap_dead start 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 shutdown_connections 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 >> 172.21.15.114:0/1279141018 conn(0x7f518c0cfa30 msgr2=0x7f518c0b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 shutdown_connections 2023-11-24T06:24:30.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.303+0000 7f5193d67700 1 -- 172.21.15.114:0/1279141018 wait complete. 2023-11-24T06:24:30.331 INFO:teuthology.orchestra.run.smithi114.stdout:55834574858 2023-11-24T06:24:30.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:24:30.332 INFO:teuthology.orchestra.run.smithi114.stdout:55834574859 2023-11-24T06:24:30.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:24:30.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.555+0000 7f5238ae8700 1 Processor -- start 2023-11-24T06:24:30.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.555+0000 7f5238ae8700 1 -- start start 2023-11-24T06:24:30.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.556+0000 7f5238ae8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f523416ae60 0x7f523406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.556+0000 7f5238ae8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f5234070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.556+0000 7f5238ae8700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523408bdb0 con 0x7f5234072bb0 2023-11-24T06:24:30.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.556+0000 7f5238ae8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523406ce30 con 0x7f523416ae60 2023-11-24T06:24:30.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.556+0000 7f5238ae8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523406d110 con 0x7f52340b4260 2023-11-24T06:24:30.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.557+0000 7f523259c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f523416ae60 0x7f523406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.557+0000 7f5232d9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f5234070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.557+0000 7f523259c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f523416ae60 0x7f523406efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33696/0 (socket says 172.21.15.114:33696) 2023-11-24T06:24:30.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.557+0000 7f523259c700 1 -- 172.21.15.114:0/1801073872 learned_addr learned my addr 172.21.15.114:0/1801073872 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.557+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986836289 0 0) 0x7f523408bdb0 con 0x7f5234072bb0 2023-11-24T06:24:30.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f521c0079c0 con 0x7f5234072bb0 2023-11-24T06:24:30.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449032786 0 0) 0x7f521c0079c0 con 0x7f5234072bb0 2023-11-24T06:24:30.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 msgr2=0x7f5234070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f5234070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 >> v2:172.21.15.172:3300/0 conn(0x7f523416ae60 msgr2=0x7f523406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 --2- 172.21.15.114:0/1801073872 >> v2:172.21.15.172:3300/0 conn(0x7f523416ae60 0x7f523406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523406d3f0 con 0x7f5234072bb0 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f5228001670 con 0x7f5234072bb0 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 >> v1:172.21.15.114:6789/0 conn(0x7f5234072bb0 legacy=0x7f52340b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f521c007e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c008390 0x7f521c008c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f521c008810 0x7f521c0091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523406ce30 con 0x7f521c008390 2023-11-24T06:24:30.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523406d110 con 0x7f521c008810 2023-11-24T06:24:30.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.558+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c009890 con 0x7f52340b4260 2023-11-24T06:24:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c008390 0x7f521c008c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5232d9d700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f521c008810 0x7f521c0091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f523259c700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f521c007e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 -- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 msgr2=0x7f521c007e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52340b4260 0x7f521c007e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 -- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f521c008810 msgr2=0x7f521c0091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f521c008810 0x7f521c0091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 -- 172.21.15.114:0/1801073872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523406d3f0 con 0x7f521c008390 2023-11-24T06:24:30.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f5231d9b700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c008390 0x7f521c008c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f522800c1e0 tx=0x7f521c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e102d158fbf4c7ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228007b70 con 0x7f521c008390 2023-11-24T06:24:30.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800cd40 con 0x7f521c008390 2023-11-24T06:24:30.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.559+0000 7f523159a700 1 -- 172.21.15.114:0/1801073872 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228012640 con 0x7f521c008390 2023-11-24T06:24:30.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.560+0000 7f5238ae8700 1 -- 172.21.15.114:0/1801073872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c008390 msgr2=0x7f521c008c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.560+0000 7f5238ae8700 1 --2- 172.21.15.114:0/1801073872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f521c008390 0x7f521c008c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f522800c1e0 tx=0x7f521c009c90 comp rx=0 tx=0).stop 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.561+0000 7f5232d9d700 1 -- 172.21.15.114:0/1801073872 reap_dead start 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.561+0000 7f5238ae8700 1 -- 172.21.15.114:0/1801073872 shutdown_connections 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.561+0000 7f5238ae8700 1 -- 172.21.15.114:0/1801073872 >> 172.21.15.114:0/1801073872 conn(0x7f52340cfa30 msgr2=0x7f52340cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.561+0000 7f5238ae8700 1 -- 172.21.15.114:0/1801073872 shutdown_connections 2023-11-24T06:24:30.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.561+0000 7f5238ae8700 1 -- 172.21.15.114:0/1801073872 wait complete. 2023-11-24T06:24:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.563+0000 7f5238ae8700 1 Processor -- start 2023-11-24T06:24:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.564+0000 7f5238ae8700 1 -- start start 2023-11-24T06:24:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 0x7f52342203b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5234223930 0x7f5234223190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.573 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340b36b0 con 0x7f52340aeb30 2023-11-24T06:24:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234223df0 con 0x7f52340ae750 2023-11-24T06:24:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5238ae8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52342240d0 con 0x7f5234223930 2023-11-24T06:24:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5232d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5232d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38052/0 (socket says 172.21.15.114:38052) 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5232d9d700 1 -- 172.21.15.114:0/177811796 learned_addr learned my addr 172.21.15.114:0/177811796 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f5231d9b700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5234223930 0x7f5234223190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 -- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5234223930 msgr2=0x7f5234223190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5234223930 0x7f5234223190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 -- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 msgr2=0x7f52342203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.566+0000 7f523259c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 0x7f52342203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 0x7f52342203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 -- 172.21.15.114:0/177811796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52342243b0 con 0x7f52340aeb30 2023-11-24T06:24:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f523259c700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 0x7f52342203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.567+0000 7f5232d9d700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5234072b60 tx=0x7f522400f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.568+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240142d0 con 0x7f52340aeb30 2023-11-24T06:24:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.568+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224014430 con 0x7f52340aeb30 2023-11-24T06:24:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.568+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240147e0 con 0x7f52340aeb30 2023-11-24T06:24:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.569+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5234224690 con 0x7f52340aeb30 2023-11-24T06:24:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.571+0000 7fbedc8c8700 1 Processor -- start 2023-11-24T06:24:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.571+0000 7fbedc8c8700 1 -- start start 2023-11-24T06:24:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbedc8c8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbed80b4260 0x7fbed806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbedc8c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8175af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbedc8c8700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed816b0a0 con 0x7fbed8072bb0 2023-11-24T06:24:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbedc8c8700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed806ce30 con 0x7fbed80b4260 2023-11-24T06:24:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbedc8c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed806d110 con 0x7fbed816abf0 2023-11-24T06:24:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbed659c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbed8072bb0 0x7fbed80b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53764/0 (socket says 172.21.15.114:53764) 2023-11-24T06:24:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7fbed659c700 1 -- 172.21.15.114:0/2530067824 learned_addr learned my addr 172.21.15.114:0/2530067824 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 --2- 172.21.15.114:0/2530067824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 -- 172.21.15.114:0/2530067824 >> v2:172.21.15.114:3300/0 conn(0x7fbed80b4260 msgr2=0x7fbed806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 --2- 172.21.15.114:0/2530067824 >> v2:172.21.15.114:3300/0 conn(0x7fbed80b4260 0x7fbed806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 -- 172.21.15.114:0/2530067824 >> v1:172.21.15.114:6789/0 conn(0x7fbed8072bb0 legacy=0x7fbed80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 -- 172.21.15.114:0/2530067824 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed806d3f0 con 0x7fbed816abf0 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7fbed6d9d700 1 --2- 172.21.15.114:0/2530067824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8175af0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbed000bfa0 tx=0x7fbed000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5cc08e5f59f901 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.569+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52342236d0 con 0x7f52340aeb30 2023-11-24T06:24:30.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.570+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5234224bb0 con 0x7f52340aeb30 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f522401a020 con 0x7f52340aeb30 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7f5222ffd700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 0x7f521406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.573+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5224089790 con 0x7f52340aeb30 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.574+0000 7f523259c700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 0x7f521406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.575+0000 7f523259c700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 0x7f521406d3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5234070b30 tx=0x7f522c00a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.577+0000 7fbed559a700 1 -- 172.21.15.114:0/2530067824 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0010070 con 0x7fbed816abf0 2023-11-24T06:24:30.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.577+0000 7fbed559a700 1 -- 172.21.15.114:0/2530067824 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed000b2e0 con 0x7fbed816abf0 2023-11-24T06:24:30.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.577+0000 7fbed559a700 1 -- 172.21.15.114:0/2530067824 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000dcf0 con 0x7fbed816abf0 2023-11-24T06:24:30.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.578+0000 7ff1089ab700 1 Processor -- start 2023-11-24T06:24:30.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.578+0000 7ff1089ab700 1 -- start start 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.577+0000 7f14b3379700 1 Processor -- start 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.577+0000 7f14b3379700 1 -- start start 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b3379700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f14ac0b3b50 0x7f14ac0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b3379700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 0x7f14ac16bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b3379700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac06ced0 con 0x7f14ac06d4d0 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b3379700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac06da70 con 0x7f14ac0b3b50 2023-11-24T06:24:30.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b3379700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0b4470 con 0x7f14ac164a90 2023-11-24T06:24:30.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7ff1089ab700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff1040b4260 0x7ff10406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7ff1089ab700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff10416abf0 0x7ff104070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7ff1089ab700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10416b0a0 con 0x7ff104072bb0 2023-11-24T06:24:30.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7ff1089ab700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10406ce30 con 0x7ff1040b4260 2023-11-24T06:24:30.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7ff1089ab700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10406d110 con 0x7ff10416abf0 2023-11-24T06:24:30.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f79c2349700 1 Processor -- start 2023-11-24T06:24:30.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f79c2349700 1 -- start start 2023-11-24T06:24:30.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff10416abf0 0x7ff104070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff10416abf0 0x7ff104070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33732/0 (socket says 172.21.15.114:33732) 2023-11-24T06:24:30.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 -- 172.21.15.114:0/2669002950 learned_addr learned my addr 172.21.15.114:0/2669002950 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff101d9b700 1 --2- 172.21.15.114:0/2669002950 >> v2:172.21.15.114:3300/0 conn(0x7ff1040b4260 0x7ff10406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 -- 172.21.15.114:0/2530067824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 msgr2=0x7fbed8175af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/2530067824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8175af0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbed000bfa0 tx=0x7fbed000d330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 -- 172.21.15.114:0/2669002950 >> v2:172.21.15.114:3300/0 conn(0x7ff1040b4260 msgr2=0x7ff10406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 --2- 172.21.15.114:0/2669002950 >> v2:172.21.15.114:3300/0 conn(0x7ff1040b4260 0x7ff10406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 -- 172.21.15.114:0/2669002950 >> v1:172.21.15.172:6789/0 conn(0x7ff104072bb0 legacy=0x7ff1040b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 -- 172.21.15.114:0/2669002950 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10406d3f0 con 0x7ff10416abf0 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b0914700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f14ac06d4d0 0x7f14ac06eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47372/0 (socket says 172.21.15.114:47372) 2023-11-24T06:24:30.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.579+0000 7f14b0914700 1 -- 172.21.15.114:0/869460078 learned_addr learned my addr 172.21.15.114:0/869460078 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 -- 172.21.15.114:0/2530067824 shutdown_connections 2023-11-24T06:24:30.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/2530067824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8175af0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/2530067824 >> v2:172.21.15.114:3300/0 conn(0x7fbed80b4260 0x7fbed806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 -- 172.21.15.114:0/2530067824 >> 172.21.15.114:0/2530067824 conn(0x7fbed80cfa30 msgr2=0x7fbed80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 -- 172.21.15.114:0/2530067824 shutdown_connections 2023-11-24T06:24:30.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7f14b1115700 1 --2- 172.21.15.114:0/869460078 >> v2:172.21.15.114:3300/0 conn(0x7f14ac0b3b50 0x7f14ac0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7fbedc8c8700 1 -- 172.21.15.114:0/2530067824 wait complete. 2023-11-24T06:24:30.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.580+0000 7ff102d9d700 1 --2- 172.21.15.114:0/2669002950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff104070d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff104062330 tx=0x7ff0fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4077f474051658 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c2349700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79bc0cf440 0x7f79bc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c2349700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c2349700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc072110 con 0x7f79bc070fd0 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c2349700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc06ee00 con 0x7f79bc0cf440 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c2349700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc071500 con 0x7f79bc1665a0 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60854/0 (socket says 172.21.15.114:60854) 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 -- 172.21.15.114:0/551248730 learned_addr learned my addr 172.21.15.114:0/551248730 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79bb7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f79bc070fd0 0x7f79bc165e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53784/0 (socket says 172.21.15.114:53784) 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79bbfff700 1 --2- 172.21.15.114:0/551248730 >> v2:172.21.15.114:3300/0 conn(0x7f79bc0cf440 0x7f79bc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 --2- 172.21.15.114:0/869460078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 0x7f14ac16bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 -- 172.21.15.114:0/869460078 >> v2:172.21.15.114:3300/0 conn(0x7f14ac0b3b50 msgr2=0x7f14ac0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 --2- 172.21.15.114:0/869460078 >> v2:172.21.15.114:3300/0 conn(0x7f14ac0b3b50 0x7f14ac0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 -- 172.21.15.114:0/869460078 >> v1:172.21.15.172:6789/0 conn(0x7f14ac06d4d0 legacy=0x7f14ac06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 -- 172.21.15.114:0/869460078 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac1634e0 con 0x7f14ac164a90 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7ff10159a700 1 -- 172.21.15.114:0/2669002950 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00e070 con 0x7ff10416abf0 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7ff10159a700 1 -- 172.21.15.114:0/2669002950 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc003a20 con 0x7ff10416abf0 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7ff10159a700 1 -- 172.21.15.114:0/2669002950 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc003db0 con 0x7ff10416abf0 2023-11-24T06:24:30.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f14b1916700 1 --2- 172.21.15.114:0/869460078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 0x7f14ac16bee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009d90 tx=0x7f14a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93864746db4c5264 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 -- 172.21.15.114:0/551248730 >> v2:172.21.15.114:3300/0 conn(0x7f79bc0cf440 msgr2=0x7f79bc06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 --2- 172.21.15.114:0/551248730 >> v2:172.21.15.114:3300/0 conn(0x7f79bc0cf440 0x7f79bc06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 -- 172.21.15.114:0/551248730 >> v1:172.21.15.114:6789/0 conn(0x7f79bc070fd0 legacy=0x7f79bc165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 -- 172.21.15.114:0/551248730 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0717e0 con 0x7f79bc1665a0 2023-11-24T06:24:30.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.581+0000 7f79c08e6700 1 --2- 172.21.15.114:0/551248730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79b00098f0 tx=0x7f79b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18230690e67fc8d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f14a3fff700 1 -- 172.21.15.114:0/869460078 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400e070 con 0x7f14ac164a90 2023-11-24T06:24:30.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f14a3fff700 1 -- 172.21.15.114:0/869460078 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400bb20 con 0x7f14ac164a90 2023-11-24T06:24:30.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f14a3fff700 1 -- 172.21.15.114:0/869460078 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4012430 con 0x7f14ac164a90 2023-11-24T06:24:30.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f79baffd700 1 -- 172.21.15.114:0/551248730 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0011830 con 0x7f79bc1665a0 2023-11-24T06:24:30.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f79baffd700 1 -- 172.21.15.114:0/551248730 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0011990 con 0x7f79bc1665a0 2023-11-24T06:24:30.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7f79baffd700 1 -- 172.21.15.114:0/551248730 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0011d20 con 0x7f79bc1665a0 2023-11-24T06:24:30.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 -- 172.21.15.114:0/2669002950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 msgr2=0x7ff104070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 --2- 172.21.15.114:0/2669002950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff104070d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff104062330 tx=0x7ff0fc00b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7fbedc8c8700 1 Processor -- start 2023-11-24T06:24:30.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 -- 172.21.15.114:0/2669002950 shutdown_connections 2023-11-24T06:24:30.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 --2- 172.21.15.114:0/2669002950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff104070d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 --2- 172.21.15.114:0/2669002950 >> v2:172.21.15.114:3300/0 conn(0x7ff1040b4260 0x7ff10406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 -- 172.21.15.114:0/2669002950 >> 172.21.15.114:0/2669002950 conn(0x7ff1040cfa30 msgr2=0x7ff1040cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7ff1089ab700 1 -- 172.21.15.114:0/2669002950 shutdown_connections 2023-11-24T06:24:30.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.582+0000 7fbedc8c8700 1 -- start start 2023-11-24T06:24:30.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 -- 172.21.15.114:0/551248730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 msgr2=0x7f79bc0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 --2- 172.21.15.114:0/551248730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79b00098f0 tx=0x7f79b0009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:30.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7ff1089ab700 1 -- 172.21.15.114:0/2669002950 wait complete. 2023-11-24T06:24:30.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 -- 172.21.15.114:0/551248730 shutdown_connections 2023-11-24T06:24:30.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 --2- 172.21.15.114:0/551248730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc1665a0 0x7f79bc0b2a40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 --2- 172.21.15.114:0/551248730 >> v2:172.21.15.114:3300/0 conn(0x7f79bc0cf440 0x7f79bc06e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 -- 172.21.15.114:0/551248730 >> 172.21.15.114:0/551248730 conn(0x7f79bc0b1740 msgr2=0x7f79bc0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 -- 172.21.15.114:0/551248730 shutdown_connections 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f79c2349700 1 -- 172.21.15.114:0/551248730 wait complete. 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 -- 172.21.15.114:0/869460078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 msgr2=0x7f14ac16bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 --2- 172.21.15.114:0/869460078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 0x7f14ac16bee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009d90 tx=0x7f14a400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 -- 172.21.15.114:0/869460078 shutdown_connections 2023-11-24T06:24:30.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 --2- 172.21.15.114:0/869460078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac164a90 0x7f14ac16bee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 --2- 172.21.15.114:0/869460078 >> v2:172.21.15.114:3300/0 conn(0x7f14ac0b3b50 0x7f14ac0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7f14b3379700 1 -- 172.21.15.114:0/869460078 >> 172.21.15.114:0/869460078 conn(0x7f14ac0cfa30 msgr2=0x7f14ac0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8208a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed8163de0 0x7fbed80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80ce940 con 0x7fbed80b4260 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed816bf40 con 0x7fbed8163de0 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.583+0000 7fbedc8c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8163950 con 0x7fbed816abf0 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38082/0 (socket says 172.21.15.114:38082) 2023-11-24T06:24:30.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 -- 172.21.15.114:0/36825215 learned_addr learned my addr 172.21.15.114:0/36825215 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed6d9d700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed8163de0 0x7fbed80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed5d9b700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8208a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 -- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 msgr2=0x7fbed8208a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8208a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 -- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed8163de0 msgr2=0x7fbed80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed8163de0 0x7fbed80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7fbed659c700 1 -- 172.21.15.114:0/36825215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed000bc80 con 0x7fbed80b4260 2023-11-24T06:24:30.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5224054920 con 0x7f52340aeb30 2023-11-24T06:24:30.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.584+0000 7ff1089ab700 1 Processor -- start 2023-11-24T06:24:30.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7f79c2349700 1 Processor -- start 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7fbed659c700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c520 tx=0x7fbecc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc005d70 con 0x7fbed80b4260 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed816b6a0 con 0x7fbed80b4260 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc0070d0 con 0x7fbed80b4260 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc007440 con 0x7fbed80b4260 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7ff1089ab700 1 -- start start 2023-11-24T06:24:30.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff10416d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff10416def0 0x7ff10416e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10416c030 con 0x7ff1040b4260 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10416b3f0 con 0x7ff10416abf0 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff1089ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10416b690 con 0x7ff10416def0 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38090/0 (socket says 172.21.15.114:38090) 2023-11-24T06:24:30.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 -- 172.21.15.114:0/1982606659 learned_addr learned my addr 172.21.15.114:0/1982606659 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.585+0000 7f79c2349700 1 -- start start 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc070fd0 0x7f79bc140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 0x7f79bc1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 0x7f79bc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc141450 con 0x7f79bc1665a0 2023-11-24T06:24:30.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1557e0 con 0x7f79bc0cf440 2023-11-24T06:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c2349700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1544e0 con 0x7f79bc070fd0 2023-11-24T06:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79bb7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 0x7f79bc1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff101d9b700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff10416d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79bb7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 0x7f79bc1552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33752/0 (socket says 172.21.15.114:33752) 2023-11-24T06:24:30.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79bb7fe700 1 -- 172.21.15.114:0/1042920193 learned_addr learned my addr 172.21.15.114:0/1042920193 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 0x7f79bc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 -- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff10416def0 msgr2=0x7ff10416e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff10416def0 0x7ff10416e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 -- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 msgr2=0x7ff10416d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff10416d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 -- 172.21.15.114:0/1982606659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009b30 con 0x7ff1040b4260 2023-11-24T06:24:30.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 -- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc070fd0 msgr2=0x7f79bc140f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:30.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc070fd0 0x7f79bc140f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 -- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 msgr2=0x7f79bc1552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 0x7f79bc1552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7f79c08e6700 1 -- 172.21.15.114:0/1042920193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0009550 con 0x7f79bc1665a0 2023-11-24T06:24:30.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7ff10259c700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c4a0 tx=0x7ff0f400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40054e0 con 0x7ff1040b4260 2023-11-24T06:24:30.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.586+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed816bc60 con 0x7fbed80b4260 2023-11-24T06:24:30.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7f79c08e6700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 0x7f79bc1524b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79b0016000 tx=0x7f79b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4003680 con 0x7ff1040b4260 2023-11-24T06:24:30.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003a30 con 0x7ff1040b4260 2023-11-24T06:24:30.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10416b970 con 0x7ff1040b4260 2023-11-24T06:24:30.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10415fb10 con 0x7ff1040b4260 2023-11-24T06:24:30.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0004450 con 0x7f79bc1665a0 2023-11-24T06:24:30.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc1547c0 con 0x7f79bc1665a0 2023-11-24T06:24:30.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.588+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc154d80 con 0x7f79bc1665a0 2023-11-24T06:24:30.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.588+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10406e300 con 0x7ff1040b4260 2023-11-24T06:24:30.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.588+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b001f070 con 0x7f79bc1665a0 2023-11-24T06:24:30.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.588+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b00038a0 con 0x7f79bc1665a0 2023-11-24T06:24:30.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.589+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0052e0 con 0x7f79bc1665a0 2023-11-24T06:24:30.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7f14b3379700 1 -- 172.21.15.114:0/869460078 shutdown_connections 2023-11-24T06:24:30.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7f14b3379700 1 -- 172.21.15.114:0/869460078 wait complete. 2023-11-24T06:24:30.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.587+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed80751e0 con 0x7fbed80b4260 2023-11-24T06:24:30.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.589+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbecc0075a0 con 0x7fbed80b4260 2023-11-24T06:24:30.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.589+0000 7fbec6ffd700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 0x7fbec006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.589+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbecc088ea0 con 0x7fbed80b4260 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7fbed5d9b700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 0x7fbec006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7fbed5d9b700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 0x7fbec006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbed8161980 tx=0x7fbec8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7f65bc0e1700 1 Processor -- start 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7f65bc0e1700 1 -- start start 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.591+0000 7f65bc0e1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f65b40adf00 0x7f65b406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.591+0000 7f65bc0e1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f65b406c540 0x7f65b40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.591+0000 7f65bc0e1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b40af950 con 0x7f65b41446a0 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.591+0000 7f65bc0e1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b406c9f0 con 0x7f65b406c540 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.591+0000 7f65bc0e1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b4144a80 con 0x7f65b40adf00 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65b9e7d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f65b41446a0 0x7f65b40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53810/0 (socket says 172.21.15.114:53810) 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65b9e7d700 1 -- 172.21.15.114:0/3951801720 learned_addr learned my addr 172.21.15.114:0/3951801720 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f14b3379700 1 Processor -- start 2023-11-24T06:24:30.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.589+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0f4003b90 con 0x7ff1040b4260 2023-11-24T06:24:30.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.590+0000 7ff0f2ffd700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 0x7ff0ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 --2- 172.21.15.114:0/3951801720 >> v2:172.21.15.114:3300/0 conn(0x7f65b406c540 0x7f65b40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 -- 172.21.15.114:0/3951801720 >> v2:172.21.15.172:3300/0 conn(0x7f65b40adf00 msgr2=0x7f65b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:30.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 --2- 172.21.15.114:0/3951801720 >> v2:172.21.15.172:3300/0 conn(0x7f65b40adf00 0x7f65b406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 -- 172.21.15.114:0/3951801720 >> v1:172.21.15.114:6789/0 conn(0x7f65b41446a0 legacy=0x7f65b40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 -- 172.21.15.114:0/3951801720 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40afc30 con 0x7f65b406c540 2023-11-24T06:24:30.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7ff101d9b700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 0x7ff0ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7f65ba67e700 1 --2- 172.21.15.114:0/3951801720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b406c540 0x7f65b40b3d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65b409f030 tx=0x7f65b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818539179cbbd7b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.592+0000 7ff101d9b700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 0x7ff0ec06d540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff10416a870 tx=0x7ff0f8009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.594+0000 7f14b3379700 1 -- start start 2023-11-24T06:24:30.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.594+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0f4003e40 con 0x7ff1040b4260 2023-11-24T06:24:30.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.594+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbecc053ff0 con 0x7fbed80b4260 2023-11-24T06:24:30.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.594+0000 7f65b8e7b700 1 -- 172.21.15.114:0/3951801720 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0011830 con 0x7f65b406c540 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14ac164a90 0x7f14ac158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 0x7f14ac200960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac202530 con 0x7f14ac0b3b50 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac2017b0 con 0x7f14ac164a90 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b3379700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac201a90 con 0x7f14ac154dd0 2023-11-24T06:24:30.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38110/0 (socket says 172.21.15.114:38110) 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 -- 172.21.15.114:0/2150326987 learned_addr learned my addr 172.21.15.114:0/2150326987 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1916700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 0x7f14ac200960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 -- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 msgr2=0x7f14ac200960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 0x7f14ac200960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 -- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14ac164a90 msgr2=0x7f14ac158a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:30.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14ac164a90 0x7f14ac158a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1115700 1 -- 172.21.15.114:0/2150326987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009a70 con 0x7f14ac0b3b50 2023-11-24T06:24:30.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f14b1916700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 0x7f14ac200960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:30.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.595+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79b0003a00 con 0x7f79bc1665a0 2023-11-24T06:24:30.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.596+0000 7f14b1115700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9b0 tx=0x7f149c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.596+0000 7f79b8ff9700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.596+0000 7f79bbfff700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.596+0000 7f79bbfff700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 0x7f79a806d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e110 tx=0x7f79b400d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.600+0000 7f23d6e5a700 1 Processor -- start 2023-11-24T06:24:30.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.600+0000 7f65b8e7b700 1 -- 172.21.15.114:0/3951801720 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b0011990 con 0x7f65b406c540 2023-11-24T06:24:30.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.600+0000 7f65b8e7b700 1 -- 172.21.15.114:0/3951801720 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0011d20 con 0x7f65b406c540 2023-11-24T06:24:30.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3951801720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b406c540 msgr2=0x7f65b40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3951801720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b406c540 0x7f65b40b3d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65b409f030 tx=0x7f65b0009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:30.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0f4053890 con 0x7ff1040b4260 2023-11-24T06:24:30.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.600+0000 7f23d6e5a700 1 -- start start 2023-11-24T06:24:30.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d6e5a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f23d00d1570 0x7f23d00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d6e5a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23d00b2fc0 0x7f23d00d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d6e5a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00d2d10 con 0x7f23d015dfc0 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d6e5a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d00cefd0 con 0x7f23d00b2fc0 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d6e5a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d00cf2b0 con 0x7f23d00d1570 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d4bf6700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f23d015dfc0 0x7f23d00d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53826/0 (socket says 172.21.15.114:53826) 2023-11-24T06:24:30.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f23d4bf6700 1 -- 172.21.15.114:0/1807475321 learned_addr learned my addr 172.21.15.114:0/1807475321 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:30.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3951801720 shutdown_connections 2023-11-24T06:24:30.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3951801720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b406c540 0x7f65b40b3d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3951801720 >> v2:172.21.15.172:3300/0 conn(0x7f65b40adf00 0x7f65b406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.601+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3951801720 >> 172.21.15.114:0/3951801720 conn(0x7f65b4071b90 msgr2=0x7f65b4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1807475321 >> v2:172.21.15.114:3300/0 conn(0x7f23d00b2fc0 0x7f23d00d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79b0021070 con 0x7f79bc1665a0 2023-11-24T06:24:30.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 -- 172.21.15.114:0/1807475321 >> v2:172.21.15.172:3300/0 conn(0x7f23d00d1570 msgr2=0x7f23d00d34d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:30.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1807475321 >> v2:172.21.15.172:3300/0 conn(0x7f23d00d1570 0x7f23d00d34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 -- 172.21.15.114:0/1807475321 >> v1:172.21.15.114:6789/0 conn(0x7f23d015dfc0 legacy=0x7f23d00d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 -- 172.21.15.114:0/1807475321 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00d2fb0 con 0x7f23d00b2fc0 2023-11-24T06:24:30.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.602+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1807475321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00b2fc0 0x7f23d00d3a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23d00b5f70 tx=0x7f23c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f593873d7ae6166 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.604+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00c850 con 0x7f14ac0b3b50 2023-11-24T06:24:30.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.604+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac201d70 con 0x7f14ac0b3b50 2023-11-24T06:24:30.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.604+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac201080 con 0x7f14ac0b3b50 2023-11-24T06:24:30.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.605+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac201fb0 con 0x7f14ac0b3b50 2023-11-24T06:24:30.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.605+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c005a20 con 0x7f14ac0b3b50 2023-11-24T06:24:30.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.605+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c005d90 con 0x7f14ac0b3b50 2023-11-24T06:24:30.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.606+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79b0055f80 con 0x7f79bc1665a0 2023-11-24T06:24:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.606+0000 7f23cf7fe700 1 -- 172.21.15.114:0/1807475321 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800e070 con 0x7f23d00b2fc0 2023-11-24T06:24:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.606+0000 7f23cf7fe700 1 -- 172.21.15.114:0/1807475321 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c800bbd0 con 0x7f23d00b2fc0 2023-11-24T06:24:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.606+0000 7f23cf7fe700 1 -- 172.21.15.114:0/1807475321 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8012430 con 0x7f23d00b2fc0 2023-11-24T06:24:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.607+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1807475321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00b2fc0 msgr2=0x7f23d00d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.607+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1807475321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00b2fc0 0x7f23d00d3a10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23d00b5f70 tx=0x7f23c800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.607+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3951801720 shutdown_connections 2023-11-24T06:24:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.607+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f149c00c9b0 con 0x7f14ac0b3b50 2023-11-24T06:24:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.608+0000 7f14a1ffb700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 0x7f149006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.608+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f149c089d00 con 0x7f14ac0b3b50 2023-11-24T06:24:30.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1807475321 shutdown_connections 2023-11-24T06:24:30.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1807475321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00b2fc0 0x7f23d00d3a10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1807475321 >> v2:172.21.15.172:3300/0 conn(0x7f23d00d1570 0x7f23d00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1807475321 >> 172.21.15.114:0/1807475321 conn(0x7f23d006b410 msgr2=0x7f23d006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.607+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3951801720 wait complete. 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f65bc0e1700 1 Processor -- start 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.609+0000 7f65bc0e1700 1 -- start start 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b406c540 0x7f65b42002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b41446a0 0x7f65b413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413cb30 con 0x7f65b40adf00 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b415d000 con 0x7f65b41446a0 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65bc0e1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413cdf0 con 0x7f65b406c540 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38126/0 (socket says 172.21.15.114:38126) 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 -- 172.21.15.114:0/3917488249 learned_addr learned my addr 172.21.15.114:0/3917488249 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65ba67e700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b41446a0 0x7f65b413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 -- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b406c540 msgr2=0x7f65b42002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b406c540 0x7f65b42002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 -- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b41446a0 msgr2=0x7f65b413d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b41446a0 0x7f65b413d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 -- 172.21.15.114:0/3917488249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00094a0 con 0x7f65b40adf00 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.611+0000 7f65b967c700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b520 tx=0x7f65a400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.612+0000 7f14b0914700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 0x7f149006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.612+0000 7f14b0914700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 0x7f149006d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac162300 tx=0x7f14a8007ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4011830 con 0x7f65b40adf00 2023-11-24T06:24:30.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b413d0d0 con 0x7f65b40adf00 2023-11-24T06:24:30.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a4011990 con 0x7f65b40adf00 2023-11-24T06:24:30.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4011d20 con 0x7f65b40adf00 2023-11-24T06:24:30.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f149c054e40 con 0x7f14ac0b3b50 2023-11-24T06:24:30.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.613+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b415c070 con 0x7f65b40adf00 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.614+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b40753f0 con 0x7f65b40adf00 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.615+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65a401a400 con 0x7f65b40adf00 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.615+0000 7f65aa7fc700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 0x7f659806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.615+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65a40895b0 con 0x7f65b40adf00 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.615+0000 7f65b9e7d700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 0x7f659806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.616+0000 7f65b9e7d700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 0x7f659806d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0012e0 tx=0x7f65ac001590 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.618+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1807475321 shutdown_connections 2023-11-24T06:24:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.618+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1807475321 wait complete. 2023-11-24T06:24:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.619+0000 7f23d6e5a700 1 Processor -- start 2023-11-24T06:24:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.619+0000 7f23d6e5a700 1 -- start start 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00b2fc0 0x7f23d00c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 0x7f23d00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 0x7f23d00b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d015ecc0 con 0x7f23d00d1570 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00c9d70 con 0x7f23d015dfc0 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.620+0000 7f23d6e5a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00c91f0 con 0x7f23d00b2fc0 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23cffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 0x7f23d00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23cffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 0x7f23d00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38136/0 (socket says 172.21.15.114:38136) 2023-11-24T06:24:30.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23cffff700 1 -- 172.21.15.114:0/1086088786 learned_addr learned my addr 172.21.15.114:0/1086088786 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 0x7f23d00b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65a40545e0 con 0x7f65b40adf00 2023-11-24T06:24:30.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 -- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00b2fc0 msgr2=0x7f23d00c6ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:30.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00b2fc0 0x7f23d00c6ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 -- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 msgr2=0x7f23d00b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 0x7f23d00b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 -- 172.21.15.114:0/1086088786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009a70 con 0x7f23d015dfc0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.621+0000 7f23d53f7700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 0x7f23d00b4a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23c8016000 tx=0x7f23c8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.622+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8012430 con 0x7f23d015dfc0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.622+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8020070 con 0x7f23d015dfc0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.622+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8004280 con 0x7f23d015dfc0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.622+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d00c94d0 con 0x7f23d015dfc0 2023-11-24T06:24:30.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.622+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00c9a90 con 0x7f23d015dfc0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.625+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d00c9660 con 0x7f23d015dfc0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.625+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f23c80043e0 con 0x7f23d015dfc0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.626+0000 7f23cd7fa700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 0x7f23b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.626+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c8022070 con 0x7f23d015dfc0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.628+0000 7f23d4bf6700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 0x7f23b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.628+0000 7f23d4bf6700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 0x7f23b406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23c40014c0 tx=0x7f23c4007a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.631+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23c80564b0 con 0x7f23d015dfc0 2023-11-24T06:24:30.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.640+0000 7ff073459700 1 Processor -- start 2023-11-24T06:24:30.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.644+0000 7ff073459700 1 -- start start 2023-11-24T06:24:30.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff06c15dfc0 0x7ff06c0d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff06c0b2a70 0x7ff06c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0d2d10 con 0x7ff06c15dfc0 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0cefd0 con 0x7ff06c0b2a70 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.645+0000 7ff073459700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0cf2b0 con 0x7ff06c0b2fc0 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.646+0000 7ff0711f5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff06c15dfc0 0x7ff06c0d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff06c15dfc0 0x7ff06c0d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38144/0 (socket says 172.21.15.114:38144) 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 -- 172.21.15.114:0/2872804403 learned_addr learned my addr 172.21.15.114:0/2872804403 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 -- 172.21.15.114:0/2872804403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 msgr2=0x7ff06c0d13a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2872804403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c0d13a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 -- 172.21.15.114:0/2872804403 >> v2:172.21.15.172:3300/0 conn(0x7ff06c0b2a70 msgr2=0x7ff06c161b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2872804403 >> v2:172.21.15.172:3300/0 conn(0x7ff06c0b2a70 0x7ff06c161b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 -- 172.21.15.114:0/2872804403 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0d2fb0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.647+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2872804403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d0e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0b2a00 tx=0x7ff05c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4a40a80a8f5925 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.648+0000 7ff063fff700 1 -- 172.21.15.114:0/2872804403 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00b870 con 0x7ff06c15dfc0 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.648+0000 7ff063fff700 1 -- 172.21.15.114:0/2872804403 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c00b9d0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.648+0000 7ff063fff700 1 -- 172.21.15.114:0/2872804403 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00bd60 con 0x7ff06c15dfc0 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.649+0000 7ff073459700 1 -- 172.21.15.114:0/2872804403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 msgr2=0x7ff06c0d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.649+0000 7ff073459700 1 --2- 172.21.15.114:0/2872804403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d0e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0b2a00 tx=0x7ff05c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 -- 172.21.15.114:0/2872804403 shutdown_connections 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 --2- 172.21.15.114:0/2872804403 >> v2:172.21.15.172:3300/0 conn(0x7ff06c0b2a70 0x7ff06c161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 --2- 172.21.15.114:0/2872804403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d0e60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 --2- 172.21.15.114:0/2872804403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 -- 172.21.15.114:0/2872804403 >> 172.21.15.114:0/2872804403 conn(0x7ff06c06b410 msgr2=0x7ff06c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.650+0000 7ff073459700 1 -- 172.21.15.114:0/2872804403 shutdown_connections 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.651+0000 7ff073459700 1 -- 172.21.15.114:0/2872804403 wait complete. 2023-11-24T06:24:30.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.652+0000 7ff073459700 1 Processor -- start 2023-11-24T06:24:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.653+0000 7ff073459700 1 -- start start 2023-11-24T06:24:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff06c0b2a70 0x7ff06c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c1eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c088360 con 0x7ff06c15dfc0 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c08afc0 con 0x7ff06c0b2a70 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff073459700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0c72d0 con 0x7ff06c0b2fc0 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.654+0000 7ff0719f6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38148/0 (socket says 172.21.15.114:38148) 2023-11-24T06:24:30.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 -- 172.21.15.114:0/2096098368 learned_addr learned my addr 172.21.15.114:0/2096098368 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0709f4700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c1eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff06c0b2a70 0x7ff06c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 -- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 msgr2=0x7ff06c1eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c1eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 -- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff06c0b2a70 msgr2=0x7ff06c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff06c0b2a70 0x7ff06c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 -- 172.21.15.114:0/2096098368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c009550 con 0x7ff06c15dfc0 2023-11-24T06:24:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.655+0000 7ff0719f6700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b4a0 tx=0x7ff06800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.659+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068011830 con 0x7ff06c15dfc0 2023-11-24T06:24:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.659+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0680119b0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.659+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068011d60 con 0x7ff06c15dfc0 2023-11-24T06:24:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.659+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c0cb4e0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.659+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c1f35d0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.663+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff06801a400 con 0x7ff06c15dfc0 2023-11-24T06:24:30.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.663+0000 7ff061ffb700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 0x7ff05006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:30.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.663+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff068089640 con 0x7ff06c15dfc0 2023-11-24T06:24:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.664+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 0x7ff05006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.665+0000 7ff0711f5700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 0x7ff05006d460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009c10 tx=0x7ff05c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:30.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.675+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0052e0 con 0x7ff06c15dfc0 2023-11-24T06:24:30.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.685+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0680546d0 con 0x7ff06c15dfc0 2023-11-24T06:24:31.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.991+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5234224bb0 con 0x7f52340aeb30 2023-11-24T06:24:31.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.997+0000 7f5222ffd700 1 -- 172.21.15.114:0/177811796 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5224057f40 con 0x7f52340aeb30 2023-11-24T06:24:31.001 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.003 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.997+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbed80b00f0 con 0x7fbed80b4260 2023-11-24T06:24:31.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:30.998+0000 7fbec6ffd700 1 -- 172.21.15.114:0/36825215 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbecc017020 con 0x7fbed80b4260 2023-11-24T06:24:31.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 msgr2=0x7f521406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 0x7f521406d3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5234070b30 tx=0x7f522c00a040 comp rx=0 tx=0).stop 2023-11-24T06:24:31.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 msgr2=0x7f52340ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5234072b60 tx=0x7f522400f880 comp rx=0 tx=0).stop 2023-11-24T06:24:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f799c005860 con 0x7f79bc1665a0 2023-11-24T06:24:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 shutdown_connections 2023-11-24T06:24:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f521406af20 0x7f521406d3e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5234070b30 tx=0x7f522c00a040 comp rx=0 tx=0).stop 2023-11-24T06:24:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52340aeb30 0x7f52340ceda0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52340ae750 0x7f52342203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 --2- 172.21.15.114:0/177811796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5234223930 0x7f5234223190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 >> 172.21.15.114:0/177811796 conn(0x7f52340cfa30 msgr2=0x7f52340b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 msgr2=0x7fbec006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 0x7fbec006d540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbed8161980 tx=0x7fbec8009190 comp rx=0 tx=0).stop 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 msgr2=0x7fbed80ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00c520 tx=0x7fbecc00c830 comp rx=0 tx=0).stop 2023-11-24T06:24:31.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 shutdown_connections 2023-11-24T06:24:31.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.001+0000 7f5238ae8700 1 -- 172.21.15.114:0/177811796 wait complete. 2023-11-24T06:24:31.009 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.003+0000 7f79b8ff9700 1 -- 172.21.15.114:0/1042920193 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79b0015090 con 0x7f79bc1665a0 2023-11-24T06:24:31.009 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:31.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 shutdown_connections 2023-11-24T06:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbec006b080 0x7fbec006d540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbed8163de0 0x7fbed80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed80b4260 0x7fbed80ce400 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 --2- 172.21.15.114:0/36825215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbed816abf0 0x7fbed8208a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.004+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 >> 172.21.15.114:0/36825215 conn(0x7fbed80cfa30 msgr2=0x7fbed806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 msgr2=0x7f79a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 0x7f79a806d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e110 tx=0x7f79b400d040 comp rx=0 tx=0).stop 2023-11-24T06:24:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 msgr2=0x7f79bc1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 0x7f79bc1524b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79b0016000 tx=0x7f79b0004750 comp rx=0 tx=0).stop 2023-11-24T06:24:31.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 shutdown_connections 2023-11-24T06:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79bc1665a0 0x7f79bc1524b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79bc070fd0 0x7f79bc140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 --2- 172.21.15.114:0/1042920193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79bc0cf440 0x7f79bc1552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 >> 172.21.15.114:0/1042920193 conn(0x7f79bc0b1740 msgr2=0x7f79bc0afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 shutdown_connections 2023-11-24T06:24:31.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.007+0000 7fbedc8c8700 1 -- 172.21.15.114:0/36825215 wait complete. 2023-11-24T06:24:31.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.008+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 shutdown_connections 2023-11-24T06:24:31.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.008+0000 7f79c2349700 1 -- 172.21.15.114:0/1042920193 wait complete. 2023-11-24T06:24:31.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.008+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f14ac202670 con 0x7f14ac0b3b50 2023-11-24T06:24:31.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.008+0000 7f14a1ffb700 1 -- 172.21.15.114:0/2150326987 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14ac202670 con 0x7f14ac0b3b50 2023-11-24T06:24:31.016 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.011+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 msgr2=0x7f149006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.011+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 0x7f149006d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac162300 tx=0x7f14a8007ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.011+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 msgr2=0x7f14ac1584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.011+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9b0 tx=0x7f149c00ed70 comp rx=0 tx=0).stop 2023-11-24T06:24:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 shutdown_connections 2023-11-24T06:24:31.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f149006aed0 0x7f149006d390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14ac154dd0 0x7f14ac200960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14ac0b3b50 0x7f14ac1584e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 --2- 172.21.15.114:0/2150326987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14ac164a90 0x7f14ac158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.012+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 >> 172.21.15.114:0/2150326987 conn(0x7f14ac0cfa30 msgr2=0x7f14ac070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.020+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff1040751e0 con 0x7ff1040b4260 2023-11-24T06:24:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.020+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 shutdown_connections 2023-11-24T06:24:31.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.020+0000 7f14b3379700 1 -- 172.21.15.114:0/2150326987 wait complete. 2023-11-24T06:24:31.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.021+0000 7ff0f2ffd700 1 -- 172.21.15.114:0/1982606659 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0f4017020 con 0x7ff1040b4260 2023-11-24T06:24:31.028 INFO:teuthology.orchestra.run.smithi114.stdout:55834574856 2023-11-24T06:24:31.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2023-11-24T06:24:31.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 msgr2=0x7ff0ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 0x7ff0ec06d540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff10416a870 tx=0x7ff0f8009210 comp rx=0 tx=0).stop 2023-11-24T06:24:31.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 msgr2=0x7ff10415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c4a0 tx=0x7ff0f400c7b0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 shutdown_connections 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff0ec06b080 0x7ff0ec06d540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff10416def0 0x7ff10416e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1040b4260 0x7ff10415f5d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 --2- 172.21.15.114:0/1982606659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff10416abf0 0x7ff10416d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.024+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 >> 172.21.15.114:0/1982606659 conn(0x7ff1040cfa30 msgr2=0x7ff1040af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.025+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 shutdown_connections 2023-11-24T06:24:31.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.025+0000 7ff1089ab700 1 -- 172.21.15.114:0/1982606659 wait complete. 2023-11-24T06:24:31.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2023-11-24T06:24:31.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2023-11-24T06:24:31.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.027+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f65b4067f60 con 0x7f65b40adf00 2023-11-24T06:24:31.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.027+0000 7f65aa7fc700 1 -- 172.21.15.114:0/3917488249 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65a4057c00 con 0x7f65b40adf00 2023-11-24T06:24:31.032 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 msgr2=0x7f659806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 0x7f659806d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0012e0 tx=0x7f65ac001590 comp rx=0 tx=0).stop 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 msgr2=0x7f65b415c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b520 tx=0x7f65a400b830 comp rx=0 tx=0).stop 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 shutdown_connections 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f659806aed0 0x7f659806d390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b41446a0 0x7f65b413d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b406c540 0x7f65b42002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 --2- 172.21.15.114:0/3917488249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65b40adf00 0x7f65b415c5f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 >> 172.21.15.114:0/3917488249 conn(0x7f65b4071b90 msgr2=0x7f65b415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 shutdown_connections 2023-11-24T06:24:31.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.031+0000 7f65bc0e1700 1 -- 172.21.15.114:0/3917488249 wait complete. 2023-11-24T06:24:31.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.035+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f23d0067f60 con 0x7f23d015dfc0 2023-11-24T06:24:31.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.036+0000 7f23cd7fa700 1 -- 172.21.15.114:0/1086088786 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23c8059ad0 con 0x7f23d015dfc0 2023-11-24T06:24:31.039 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2023-11-24T06:24:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 msgr2=0x7f23b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 0x7f23b406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23c40014c0 tx=0x7f23c4007a50 comp rx=0 tx=0).stop 2023-11-24T06:24:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 msgr2=0x7f23d00b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 0x7f23d00b4a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23c8016000 tx=0x7f23c8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 shutdown_connections 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23b406aed0 0x7f23b406d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23d015dfc0 0x7f23d00b4a70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23d00b2fc0 0x7f23d00c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 --2- 172.21.15.114:0/1086088786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23d00d1570 0x7f23d00b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.038+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 >> 172.21.15.114:0/1086088786 conn(0x7f23d006b410 msgr2=0x7f23d00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.039+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 shutdown_connections 2023-11-24T06:24:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.039+0000 7f23d6e5a700 1 -- 172.21.15.114:0/1086088786 wait complete. 2023-11-24T06:24:31.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.2 2023-11-24T06:24:31.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2023-11-24T06:24:31.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2023-11-24T06:24:31.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.066+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff04c005590 con 0x7ff06c15dfc0 2023-11-24T06:24:31.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.066+0000 7ff061ffb700 1 -- 172.21.15.114:0/2096098368 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff068015030 con 0x7ff06c15dfc0 2023-11-24T06:24:31.070 INFO:teuthology.orchestra.run.smithi114.stdout:55834574855 2023-11-24T06:24:31.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 msgr2=0x7ff05006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 0x7ff05006d460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009c10 tx=0x7ff05c008030 comp rx=0 tx=0).stop 2023-11-24T06:24:31.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 msgr2=0x7ff06c0d41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:31.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b4a0 tx=0x7ff06800b860 comp rx=0 tx=0).stop 2023-11-24T06:24:31.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 shutdown_connections 2023-11-24T06:24:31.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff05006afa0 0x7ff05006d460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff06c15dfc0 0x7ff06c0d41b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff06c0b2a70 0x7ff06c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 --2- 172.21.15.114:0/2096098368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff06c0b2fc0 0x7ff06c1eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:31.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 >> 172.21.15.114:0/2096098368 conn(0x7ff06c06b410 msgr2=0x7ff06c0b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:31.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 shutdown_connections 2023-11-24T06:24:31.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:31.068+0000 7ff073459700 1 -- 172.21.15.114:0/2096098368 wait complete. 2023-11-24T06:24:31.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2023-11-24T06:24:32.029 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:24:32.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:24:32.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:24:32.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:24:32.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:24:32.053 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:24:32.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:24:32.085 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.214+0000 7fd3af42c700 1 Processor -- start 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.215+0000 7fd3af42c700 1 -- start start 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3af42c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd3a8072bb0 0x7fd3a806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3af42c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3af42c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a808bdb0 con 0x7fd3a816ae60 2023-11-24T06:24:32.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3af42c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a806ce30 con 0x7fd3a8072bb0 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3af42c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806d110 con 0x7fd3a80b4260 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3ad1c8700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd3a816ae60 0x7fd3a80b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53864/0 (socket says 172.21.15.114:53864) 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.216+0000 7fd3ad1c8700 1 -- 172.21.15.114:0/1706654462 learned_addr learned my addr 172.21.15.114:0/1706654462 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.217+0000 7fd3ad9c9700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.217+0000 7fd3ac9c7700 1 --2- 172.21.15.114:0/1706654462 >> v2:172.21.15.114:3300/0 conn(0x7fd3a8072bb0 0x7fd3a806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.218+0000 7fd3ac9c7700 1 -- 172.21.15.114:0/1706654462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 msgr2=0x7fd3a8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.218+0000 7fd3ac9c7700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.218+0000 7fd3ac9c7700 1 -- 172.21.15.114:0/1706654462 >> v1:172.21.15.114:6789/0 conn(0x7fd3a816ae60 legacy=0x7fd3a80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.218+0000 7fd3ac9c7700 1 -- 172.21.15.114:0/1706654462 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a806d3f0 con 0x7fd3a8072bb0 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.218+0000 7fd3ac9c7700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8072bb0 0x7fd3a806efc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd398009d90 tx=0x7fd39800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a0aee52df36fbc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.217+0000 7fd39ffff700 1 -- 172.21.15.114:0/1706654462 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983858693 0 0) 0x7fd3a808bdb0 con 0x7fd3a816ae60 2023-11-24T06:24:32.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.219+0000 7fd39ffff700 1 -- 172.21.15.114:0/1706654462 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e070 con 0x7fd3a8072bb0 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.219+0000 7fd39ffff700 1 -- 172.21.15.114:0/1706654462 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800bbd0 con 0x7fd3a8072bb0 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.219+0000 7fd39ffff700 1 -- 172.21.15.114:0/1706654462 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398012430 con 0x7fd3a8072bb0 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.220+0000 7fd3af42c700 1 -- 172.21.15.114:0/1706654462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8072bb0 msgr2=0x7fd3a806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.220+0000 7fd3af42c700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8072bb0 0x7fd3a806efc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd398009d90 tx=0x7fd39800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 -- 172.21.15.114:0/1706654462 shutdown_connections 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 --2- 172.21.15.114:0/1706654462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8072bb0 0x7fd3a806efc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 -- 172.21.15.114:0/1706654462 >> 172.21.15.114:0/1706654462 conn(0x7fd3a80cfa30 msgr2=0x7fd3a80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 -- 172.21.15.114:0/1706654462 shutdown_connections 2023-11-24T06:24:32.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.221+0000 7fd3af42c700 1 -- 172.21.15.114:0/1706654462 wait complete. 2023-11-24T06:24:32.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.223+0000 7fd3af42c700 1 Processor -- start 2023-11-24T06:24:32.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.223+0000 7fd3af42c700 1 -- start start 2023-11-24T06:24:32.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3a8162b50 0x7fd3a8200df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8160e30 con 0x7fd3a8201330 2023-11-24T06:24:32.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8161110 con 0x7fd3a8162b50 2023-11-24T06:24:32.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3af42c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a81613f0 con 0x7fd3a80b4260 2023-11-24T06:24:32.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3ad9c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3ad9c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38160/0 (socket says 172.21.15.114:38160) 2023-11-24T06:24:32.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.224+0000 7fd3ad9c9700 1 -- 172.21.15.114:0/217903363 learned_addr learned my addr 172.21.15.114:0/217903363 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ac9c7700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3a8162b50 0x7fd3a8200df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 -- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 msgr2=0x7fd3a8162610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:32.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8162610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 -- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3a8162b50 msgr2=0x7fd3a8200df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3a8162b50 0x7fd3a8200df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 -- 172.21.15.114:0/217903363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009a70 con 0x7fd3a8201330 2023-11-24T06:24:32.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.225+0000 7fd3ad9c9700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0007e30 tx=0x7fd3a000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.226+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000ede0 con 0x7fd3a8201330 2023-11-24T06:24:32.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.226+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a0014420 con 0x7fd3a8201330 2023-11-24T06:24:32.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.226+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a00147d0 con 0x7fd3a8201330 2023-11-24T06:24:32.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.226+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a81616d0 con 0x7fd3a8201330 2023-11-24T06:24:32.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.226+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a82038d0 con 0x7fd3a8201330 2023-11-24T06:24:32.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.229+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3a0014930 con 0x7fd3a8201330 2023-11-24T06:24:32.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.229+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8161db0 con 0x7fd3a8201330 2023-11-24T06:24:32.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.229+0000 7fd39dffb700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 0x7fd38c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.229+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3a0089d10 con 0x7fd3a8201330 2023-11-24T06:24:32.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.234+0000 7f2725a85700 1 Processor -- start 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.234+0000 7f2725a85700 1 -- start start 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27200b0ba0 0x7f27200af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f27200b0f80 0x7f27200cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27200b2bd0 con 0x7f27200b0f80 2023-11-24T06:24:32.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27200b1fc0 con 0x7f27200b0ba0 2023-11-24T06:24:32.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.235+0000 7f2725a85700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27200b22a0 con 0x7f272015ee90 2023-11-24T06:24:32.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.231+0000 7fd3ad1c8700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 0x7fd38c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.231+0000 7fd3ad1c8700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 0x7fd38c06d490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a4007a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.236+0000 7f271ffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.236+0000 7f271ffff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:60946/0 (socket says 172.21.15.114:60946) 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.236+0000 7f271ffff700 1 -- 172.21.15.114:0/2223476794 learned_addr learned my addr 172.21.15.114:0/2223476794 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.236+0000 7f271f7fe700 1 --2- 172.21.15.114:0/2223476794 >> v2:172.21.15.172:3300/0 conn(0x7f27200b0ba0 0x7f27200af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 -- 172.21.15.114:0/2223476794 >> v2:172.21.15.172:3300/0 conn(0x7f27200b0ba0 msgr2=0x7f27200af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 --2- 172.21.15.114:0/2223476794 >> v2:172.21.15.172:3300/0 conn(0x7f27200b0ba0 0x7f27200af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 -- 172.21.15.114:0/2223476794 >> v2:172.21.15.114:3300/0 conn(0x7f27200b0f80 msgr2=0x7f27200cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 --2- 172.21.15.114:0/2223476794 >> v2:172.21.15.114:3300/0 conn(0x7f27200b0f80 0x7f27200cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 -- 172.21.15.114:0/2223476794 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27200b2580 con 0x7f272015ee90 2023-11-24T06:24:32.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271ffff700 1 --2- 172.21.15.114:0/2223476794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27200d1340 tx=0x7f271800d430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4544db6e4263420 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271e7fc700 1 -- 172.21.15.114:0/2223476794 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718010070 con 0x7f272015ee90 2023-11-24T06:24:32.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271e7fc700 1 -- 172.21.15.114:0/2223476794 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271800dad0 con 0x7f272015ee90 2023-11-24T06:24:32.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.237+0000 7f271e7fc700 1 -- 172.21.15.114:0/2223476794 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271800de80 con 0x7f272015ee90 2023-11-24T06:24:32.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 -- 172.21.15.114:0/2223476794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 msgr2=0x7f27200ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 --2- 172.21.15.114:0/2223476794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f27200d1340 tx=0x7f271800d430 comp rx=0 tx=0).stop 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 -- 172.21.15.114:0/2223476794 shutdown_connections 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 --2- 172.21.15.114:0/2223476794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272015ee90 0x7f27200ccca0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 --2- 172.21.15.114:0/2223476794 >> v2:172.21.15.172:3300/0 conn(0x7f27200b0ba0 0x7f27200af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.238+0000 7f2725a85700 1 --2- 172.21.15.114:0/2223476794 >> v2:172.21.15.114:3300/0 conn(0x7f27200b0f80 0x7f27200cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.239+0000 7f2725a85700 1 -- 172.21.15.114:0/2223476794 >> 172.21.15.114:0/2223476794 conn(0x7f27200b57b0 msgr2=0x7f27200b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.239+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3a0054da0 con 0x7fd3a8201330 2023-11-24T06:24:32.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.240+0000 7f2725a85700 1 -- 172.21.15.114:0/2223476794 shutdown_connections 2023-11-24T06:24:32.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.240+0000 7f2725a85700 1 -- 172.21.15.114:0/2223476794 wait complete. 2023-11-24T06:24:32.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.242+0000 7f2725a85700 1 Processor -- start 2023-11-24T06:24:32.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.243+0000 7f2725a85700 1 -- start start 2023-11-24T06:24:32.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27200b0ba0 0x7f2720200c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 0x7f27200b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27202011b0 con 0x7f27200b0f80 2023-11-24T06:24:32.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272015b420 con 0x7f272015ee90 2023-11-24T06:24:32.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f2725a85700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272015b700 con 0x7f27200b0ba0 2023-11-24T06:24:32.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f271effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f271effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38176/0 (socket says 172.21.15.114:38176) 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f271effd700 1 -- 172.21.15.114:0/121834393 learned_addr learned my addr 172.21.15.114:0/121834393 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.244+0000 7f271ffff700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 0x7f27200b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 -- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27200b0ba0 msgr2=0x7f2720200c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27200b0ba0 0x7f2720200c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 -- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 msgr2=0x7f27200b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 0x7f27200b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 -- 172.21.15.114:0/121834393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271800bd40 con 0x7f27200b0f80 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271ffff700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 0x7f27200b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7f271effd700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f271000b530 tx=0x7f271000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7efd5e4eb700 1 Processor -- start 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.245+0000 7efd5e4eb700 1 -- start start 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd5e4eb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efd5816ae60 0x7efd5806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd5e4eb700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5808bdb0 con 0x7efd580b4260 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd5e4eb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5806ce30 con 0x7efd58072bb0 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd5e4eb700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5806d110 con 0x7efd5816ae60 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd57fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efd5816ae60 0x7efd5806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd57fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efd5816ae60 0x7efd5806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38180/0 (socket says 172.21.15.114:38180) 2023-11-24T06:24:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.246+0000 7efd57fff700 1 -- 172.21.15.114:0/4197639385 learned_addr learned my addr 172.21.15.114:0/4197639385 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.249+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710011830 con 0x7f27200b0f80 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27100119b0 con 0x7f27200b0f80 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710011d60 con 0x7f27200b0f80 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7efd57fff700 1 -- 172.21.15.114:0/4197639385 >> v1:172.21.15.172:6789/0 conn(0x7efd58072bb0 legacy=0x7efd580b3b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7efd57fff700 1 -- 172.21.15.114:0/4197639385 >> v1:172.21.15.114:6789/0 conn(0x7efd580b4260 legacy=0x7efd58070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7efd57fff700 1 -- 172.21.15.114:0/4197639385 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5806d3f0 con 0x7efd5816ae60 2023-11-24T06:24:32.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.250+0000 7efd57fff700 1 --2- 172.21.15.114:0/4197639385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd5816ae60 0x7efd5806efc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd48009d50 tx=0x7efd4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b4e0d1555861c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.252+0000 7efd56ffd700 1 -- 172.21.15.114:0/4197639385 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48009ea0 con 0x7efd5816ae60 2023-11-24T06:24:32.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.252+0000 7efd56ffd700 1 -- 172.21.15.114:0/4197639385 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480092e0 con 0x7efd5816ae60 2023-11-24T06:24:32.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.252+0000 7efd56ffd700 1 -- 172.21.15.114:0/4197639385 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800e070 con 0x7efd5816ae60 2023-11-24T06:24:32.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.252+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272015b9e0 con 0x7f27200b0f80 2023-11-24T06:24:32.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.252+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272015bfa0 con 0x7f27200b0f80 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.253+0000 7efd5e4eb700 1 -- 172.21.15.114:0/4197639385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd5816ae60 msgr2=0x7efd5806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.253+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/4197639385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd5816ae60 0x7efd5806efc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd48009d50 tx=0x7efd4800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.255+0000 7efd5e4eb700 1 -- 172.21.15.114:0/4197639385 shutdown_connections 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.255+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/4197639385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd5816ae60 0x7efd5806efc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.255+0000 7efd5e4eb700 1 -- 172.21.15.114:0/4197639385 >> 172.21.15.114:0/4197639385 conn(0x7efd580cfa30 msgr2=0x7efd580cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.256+0000 7efd5e4eb700 1 -- 172.21.15.114:0/4197639385 shutdown_connections 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.256+0000 7efd5e4eb700 1 -- 172.21.15.114:0/4197639385 wait complete. 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.258+0000 7efd5e4eb700 1 Processor -- start 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.258+0000 7efd5e4eb700 1 -- start start 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 0x7efd580ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 0x7efd580ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd580cee90 0x7efd58169380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580cf310 con 0x7efd580b4260 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580abe40 con 0x7efd580cee90 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd5e4eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd580ac0b0 con 0x7efd580aee30 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd57fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 0x7efd580ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd57fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 0x7efd580ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38192/0 (socket says 172.21.15.114:38192) 2023-11-24T06:24:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd57fff700 1 -- 172.21.15.114:0/1491792813 learned_addr learned my addr 172.21.15.114:0/1491792813 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7efd577fe700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 0x7efd580ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd5ca88700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd580cee90 0x7efd58169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 -- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd580cee90 msgr2=0x7efd58169380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd580cee90 0x7efd58169380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 -- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 msgr2=0x7efd580ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 0x7efd580ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 -- 172.21.15.114:0/1491792813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48009a30 con 0x7efd580aee30 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.260+0000 7efd577fe700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 0x7efd580ce950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006fd0 tx=0x7efd4c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.261+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c011830 con 0x7efd580aee30 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.261+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c011990 con 0x7efd580aee30 2023-11-24T06:24:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.261+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c011d40 con 0x7efd580aee30 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.255+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f271001a400 con 0x7f27200b0f80 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.256+0000 7f2707fff700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 0x7f270806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.256+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2710089740 con 0x7f27200b0f80 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.256+0000 7f271f7fe700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 0x7f270806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.257+0000 7f271f7fe700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 0x7f270806d580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2714006b20 tx=0x7f2714006ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.259+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2720067f60 con 0x7f27200b0f80 2023-11-24T06:24:32.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.262+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd580ac390 con 0x7efd580aee30 2023-11-24T06:24:32.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.262+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd58165c60 con 0x7efd580aee30 2023-11-24T06:24:32.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.265+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380052e0 con 0x7efd580aee30 2023-11-24T06:24:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.269+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd4c01a400 con 0x7efd580aee30 2023-11-24T06:24:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.270+0000 7efd54ff9700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 0x7efd4006d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.270+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd4c089990 con 0x7efd580aee30 2023-11-24T06:24:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.273+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27100547d0 con 0x7f27200b0f80 2023-11-24T06:24:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.274+0000 7efd57fff700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 0x7efd4006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.275+0000 7efd57fff700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 0x7efd4006d580 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd48014040 tx=0x7efd48009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.277+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd4c054b20 con 0x7efd580aee30 2023-11-24T06:24:32.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.281+0000 7f8c1ddce700 1 Processor -- start 2023-11-24T06:24:32.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.281+0000 7f8c1ddce700 1 -- start start 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.282+0000 7f8c1ddce700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8c18072bb0 0x7f8c1806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.282+0000 7f8c1ddce700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c180b4260 0x7f8c18070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.282+0000 7f8c1ddce700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1808bdb0 con 0x7f8c1816ae60 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.282+0000 7f8c1ddce700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1806ce30 con 0x7f8c18072bb0 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.282+0000 7f8c1ddce700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1806d110 con 0x7f8c180b4260 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c177fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8c1816ae60 0x7f8c180b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53904/0 (socket says 172.21.15.114:53904) 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c177fe700 1 -- 172.21.15.114:0/2972126163 learned_addr learned my addr 172.21.15.114:0/2972126163 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281110024 0 0) 0x7f8c1808bdb0 con 0x7f8c1816ae60 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c00003590 con 0x7f8c1816ae60 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1444097004 0 0) 0x7f8c00003590 con 0x7f8c1816ae60 2023-11-24T06:24:32.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 >> v2:172.21.15.172:3300/0 conn(0x7f8c180b4260 msgr2=0x7f8c18070d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:32.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 --2- 172.21.15.114:0/2972126163 >> v2:172.21.15.172:3300/0 conn(0x7f8c180b4260 0x7f8c18070d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 >> v2:172.21.15.114:3300/0 conn(0x7f8c18072bb0 msgr2=0x7f8c1806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 --2- 172.21.15.114:0/2972126163 >> v2:172.21.15.114:3300/0 conn(0x7f8c18072bb0 0x7f8c1806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1806d3f0 con 0x7f8c1816ae60 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f8c100019f0 con 0x7f8c1816ae60 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 >> v1:172.21.15.114:6789/0 conn(0x7f8c1816ae60 legacy=0x7f8c180b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c000043c0 0x7f8c00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 0x7f8c00005480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005c80 0x7f8c0000c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1806ce30 con 0x7f8c00005c80 2023-11-24T06:24:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1806d110 con 0x7f8c00005000 2023-11-24T06:24:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.283+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0000cb70 con 0x7f8c000043c0 2023-11-24T06:24:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 0x7f8c00005480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c17fff700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005c80 0x7f8c0000c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7fe6dbaf1700 1 Processor -- start 2023-11-24T06:24:32.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 -- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c000043c0 msgr2=0x7f8c00004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c000043c0 0x7f8c00004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 -- 172.21.15.114:0/2972126163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005c80 msgr2=0x7f8c0000c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005c80 0x7f8c0000c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 -- 172.21.15.114:0/2972126163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00003590 con 0x7f8c00005000 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c17fff700 1 -- 172.21.15.114:0/2972126163 reap_dead start 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 0x7f8c00005480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb90 tx=0x7f8c00003840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e0d5a06c4f4859 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00db60 con 0x7f8c00005000 2023-11-24T06:24:32.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00dcc0 con 0x7f8c00005000 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c167fc700 1 -- 172.21.15.114:0/2972126163 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c004270 con 0x7f8c00005000 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c1ddce700 1 -- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 msgr2=0x7f8c00005480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 0x7f8c00005480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb90 tx=0x7f8c00003840 comp rx=0 tx=0).stop 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c1ddce700 1 -- 172.21.15.114:0/2972126163 shutdown_connections 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/2972126163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c00005000 0x7f8c00005480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7f8c1ddce700 1 -- 172.21.15.114:0/2972126163 >> 172.21.15.114:0/2972126163 conn(0x7f8c180cfa30 msgr2=0x7f8c180cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.284+0000 7fe6dbaf1700 1 -- start start 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.285+0000 7fe6dbaf1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6d415d6a0 0x7fe6d40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6dbaf1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d40720e0 con 0x7fe6d4071be0 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6dbaf1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d40723c0 con 0x7fe6d40b4090 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6dbaf1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4072d60 con 0x7fe6d415d6a0 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d988d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6d415d6a0 0x7fe6d40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d908c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe6d40b4090 0x7fe6d40714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47514/0 (socket says 172.21.15.114:47514) 2023-11-24T06:24:32.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d908c700 1 -- 172.21.15.114:0/1870596174 learned_addr learned my addr 172.21.15.114:0/1870596174 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:32.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7f8c1ddce700 1 -- 172.21.15.114:0/2972126163 shutdown_connections 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086474361 0 0) 0x7fe6d40723c0 con 0x7fe6d40b4090 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6c00046c0 con 0x7fe6d40b4090 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004157399 0 0) 0x7fe6d40720e0 con 0x7fe6d4071be0 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d40723c0 con 0x7fe6d4071be0 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109735935 0 0) 0x7fe6c00046c0 con 0x7fe6d40b4090 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6d415d6a0 msgr2=0x7fe6d40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6d415d6a0 0x7fe6d40b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.286+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 >> v1:172.21.15.114:6789/0 conn(0x7fe6d4071be0 legacy=0x7fe6d4140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40726a0 con 0x7fe6d40b4090 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fe6d0001670 con 0x7fe6d40b4090 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 >> v1:172.21.15.172:6789/0 conn(0x7fe6d40b4090 legacy=0x7fe6d40714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0004ca0 0x7fe6c00050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6c0005960 0x7fe6c0005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6c0006320 0x7fe6c000cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d4072d60 con 0x7fe6c0006320 2023-11-24T06:24:32.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c000d110 con 0x7fe6c0005960 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c000d2a0 con 0x7fe6c0004ca0 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6c0006320 0x7fe6c000cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 -- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0004ca0 msgr2=0x7fe6c00050a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0004ca0 0x7fe6c00050a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 -- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6c0005960 msgr2=0x7fe6c0005de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6c0005960 0x7fe6c0005de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 -- 172.21.15.114:0/1870596174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40726a0 con 0x7fe6c0006320 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6da08e700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6c0006320 0x7fe6c000cb20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c320 tx=0x7fe6c0006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea65b0e14aec069b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.287+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc00ce40 con 0x7fe6c0006320 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.288+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc003a00 con 0x7fe6c0006320 2023-11-24T06:24:32.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.288+0000 7fe6d888b700 1 -- 172.21.15.114:0/1870596174 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc003db0 con 0x7fe6c0006320 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.289+0000 7f8c1ddce700 1 -- 172.21.15.114:0/2972126163 wait complete. 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.291+0000 7f8c1ddce700 1 Processor -- start 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.291+0000 7f8c1ddce700 1 -- start start 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c18166910 0x7f8c18166d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c181652b0 0x7f8c18165760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18166330 con 0x7f8c00005000 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c181672d0 con 0x7f8c18166910 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c1ddce700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1808aef0 con 0x7f8c181652b0 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38218/0 (socket says 172.21.15.114:38218) 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 -- 172.21.15.114:0/3717480769 learned_addr learned my addr 172.21.15.114:0/3717480769 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c17fff700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c181652b0 0x7f8c18165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 -- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c181652b0 msgr2=0x7f8c18165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c181652b0 0x7f8c18165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 -- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c18166910 msgr2=0x7f8c18166d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c18166910 0x7f8c18166d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 -- 172.21.15.114:0/3717480769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180cc440 con 0x7f8c00005000 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.293+0000 7f8c177fe700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c10002150 tx=0x7f8c1000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000ae00 con 0x7f8c00005000 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100070d0 con 0x7f8c00005000 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10007480 con 0x7f8c00005000 2023-11-24T06:24:32.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c18166050 con 0x7f8c00005000 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c180cc0d0 con 0x7f8c00005000 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/1870596174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6c0006320 msgr2=0x7fe6c000cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.294+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/1870596174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6c0006320 0x7fe6c000cb20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c320 tx=0x7fe6c0006990 comp rx=0 tx=0).stop 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.295+0000 7fe6da08e700 1 -- 172.21.15.114:0/1870596174 reap_dead start 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.295+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/1870596174 shutdown_connections 2023-11-24T06:24:32.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.295+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/1870596174 >> 172.21.15.114:0/1870596174 conn(0x7fe6d40cfa30 msgr2=0x7fe6d40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.295+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c180cbc50 con 0x7f8c00005000 2023-11-24T06:24:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.296+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c100075e0 con 0x7f8c00005000 2023-11-24T06:24:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.297+0000 7f8bfffff700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 0x7f8c0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.297+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c100894b0 con 0x7f8c00005000 2023-11-24T06:24:32.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.300+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 0x7f8c0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.300+0000 7f8c16ffd700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 0x7f8c0406d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c18156d50 tx=0x7f8c0c013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.301+0000 7fc64b434700 1 Processor -- start 2023-11-24T06:24:32.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.302+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c10054560 con 0x7f8c00005000 2023-11-24T06:24:32.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.301+0000 7fc64b434700 1 -- start start 2023-11-24T06:24:32.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.302+0000 7fc64b434700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc644163480 0x7fc6440b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.302+0000 7fc64b434700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6440b2e80 con 0x7fc644163860 2023-11-24T06:24:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.302+0000 7fc64b434700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc644164510 con 0x7fc6440b2550 2023-11-24T06:24:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.302+0000 7fc64b434700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6440b2980 con 0x7fc644163480 2023-11-24T06:24:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.303+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/1870596174 shutdown_connections 2023-11-24T06:24:32.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.303+0000 7fc6489cf700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc644163860 0x7fc6440b1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53938/0 (socket says 172.21.15.114:53938) 2023-11-24T06:24:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.303+0000 7fc6489cf700 1 -- 172.21.15.114:0/710805226 learned_addr learned my addr 172.21.15.114:0/710805226 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.306+0000 7fc6491d0700 1 --2- 172.21.15.114:0/710805226 >> v2:172.21.15.172:3300/0 conn(0x7fc644163480 0x7fc6440b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.306+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425648145 0 0) 0x7fc6440b2e80 con 0x7fc644163860 2023-11-24T06:24:32.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.306+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6300035b0 con 0x7fc644163860 2023-11-24T06:24:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129186033 0 0) 0x7fc6300035b0 con 0x7fc644163860 2023-11-24T06:24:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 >> v2:172.21.15.172:3300/0 conn(0x7fc644163480 msgr2=0x7fc6440b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 --2- 172.21.15.114:0/710805226 >> v2:172.21.15.172:3300/0 conn(0x7fc644163480 0x7fc6440b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 >> v1:172.21.15.172:6789/0 conn(0x7fc6440b2550 legacy=0x7fc6440af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644160610 con 0x7fc644163860 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fc640001670 con 0x7fc644163860 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 >> v1:172.21.15.114:6789/0 conn(0x7fc644163860 legacy=0x7fc6440b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc630003a00 0x7fc630003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630004850 0x7fc630004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc630005210 0x7fc63000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440b2980 con 0x7fc630003a00 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63000c140 con 0x7fc630004850 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63000c300 con 0x7fc630005210 2023-11-24T06:24:32.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/1870596174 wait complete. 2023-11-24T06:24:32.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.307+0000 7fc6491d0700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc630003a00 0x7fc630003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 -- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc630005210 msgr2=0x7fc63000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:24:32.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc630005210 0x7fc63000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 -- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630004850 msgr2=0x7fc630004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:32.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630004850 0x7fc630004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 -- 172.21.15.114:0/710805226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644160610 con 0x7fc630003a00 2023-11-24T06:24:32.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc6491d0700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc630003a00 0x7fc630003e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b40 tx=0x7fc630005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c203cdaa08fed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.309+0000 7fe6dbaf1700 1 Processor -- start 2023-11-24T06:24:32.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634004020 con 0x7fc630003a00 2023-11-24T06:24:32.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634004180 con 0x7fc630003a00 2023-11-24T06:24:32.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.308+0000 7fc63bfff700 1 -- 172.21.15.114:0/710805226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340044f0 con 0x7fc630003a00 2023-11-24T06:24:32.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fc64b434700 1 -- 172.21.15.114:0/710805226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc630003a00 msgr2=0x7fc630003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fc64b434700 1 --2- 172.21.15.114:0/710805226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc630003a00 0x7fc630003e00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b40 tx=0x7fc630005880 comp rx=0 tx=0).stop 2023-11-24T06:24:32.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fc6499d1700 1 -- 172.21.15.114:0/710805226 reap_dead start 2023-11-24T06:24:32.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fc64b434700 1 -- 172.21.15.114:0/710805226 shutdown_connections 2023-11-24T06:24:32.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fc64b434700 1 -- 172.21.15.114:0/710805226 >> 172.21.15.114:0/710805226 conn(0x7fc6440b57b0 msgr2=0x7fc6440b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.309+0000 7fe6dbaf1700 1 -- start start 2023-11-24T06:24:32.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fc64b434700 1 -- 172.21.15.114:0/710805226 shutdown_connections 2023-11-24T06:24:32.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fc64b434700 1 -- 172.21.15.114:0/710805226 wait complete. 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.313+0000 7fc64b434700 1 Processor -- start 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.310+0000 7fe6dbaf1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0006320 0x7fe6d414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6dbaf1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 0x7fe6d414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6dbaf1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 0x7fe6d41662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6dbaf1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d41504b0 con 0x7fe6d4150030 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6dbaf1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d414ef90 con 0x7fe6d414ea30 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6dbaf1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d414f270 con 0x7fe6c0006320 2023-11-24T06:24:32.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6d908c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 0x7fe6d414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6d988d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0006320 0x7fe6d414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6d908c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 0x7fe6d414e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38236/0 (socket says 172.21.15.114:38236) 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6d908c700 1 -- 172.21.15.114:0/3638137032 learned_addr learned my addr 172.21.15.114:0/3638137032 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6da08e700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 0x7fe6d41662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6da08e700 1 -- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0006320 msgr2=0x7fe6d414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6da08e700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0006320 0x7fe6d414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6da08e700 1 -- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 msgr2=0x7fe6d414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.311+0000 7fe6da08e700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 0x7fe6d414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6da08e700 1 -- 172.21.15.114:0/3638137032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d414f550 con 0x7fe6d414ea30 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6da08e700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 0x7fe6d41662f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc014000 tx=0x7fe6cc001650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0011c0 con 0x7fe6d414ea30 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0052e0 con 0x7fe6d414ea30 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc005690 con 0x7fe6d414ea30 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d414f830 con 0x7fe6d414ea30 2023-11-24T06:24:32.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.312+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d4152cb0 con 0x7fe6d414ea30 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.314+0000 7fc64b434700 1 -- start start 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6441535d0 0x7fc6440b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6440b4670 0x7fc6440b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440b33c0 con 0x7fc6441539b0 2023-11-24T06:24:32.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440b35d0 con 0x7fc6440b4670 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.315+0000 7fc64b434700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440b38b0 con 0x7fc6441535d0 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38244/0 (socket says 172.21.15.114:38244) 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 -- 172.21.15.114:0/592730133 learned_addr learned my addr 172.21.15.114:0/592730133 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 -- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6441535d0 msgr2=0x7fc6440b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6441535d0 0x7fc6440b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 -- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6440b4670 msgr2=0x7fc6440b2e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:32.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6440b4670 0x7fc6440b2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fc6499d1700 1 -- 172.21.15.114:0/592730133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440b3b90 con 0x7fc6441539b0 2023-11-24T06:24:32.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.317+0000 7fc6499d1700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ce310 tx=0x7fc63c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.316+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6cc01d420 con 0x7fe6d414ea30 2023-11-24T06:24:32.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.317+0000 7fe6ca7fc700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 0x7fe6b808f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.317+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6cc08a560 con 0x7fe6d414ea30 2023-11-24T06:24:32.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.317+0000 7fe6d988d700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 0x7fe6b808f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.318+0000 7fe6d988d700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 0x7fe6b808f470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40737a0 tx=0x7fe6c4009aa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.319+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00a520 con 0x7fc6441539b0 2023-11-24T06:24:32.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.319+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c00a680 con 0x7fc6441539b0 2023-11-24T06:24:32.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.319+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00aa10 con 0x7fc6441539b0 2023-11-24T06:24:32.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.320+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6440b3e70 con 0x7fc6441539b0 2023-11-24T06:24:32.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.320+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6440ad7b0 con 0x7fc6441539b0 2023-11-24T06:24:32.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.323+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc0052e0 con 0x7fe6d414ea30 2023-11-24T06:24:32.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.323+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6440b39f0 con 0x7fc6441539b0 2023-11-24T06:24:32.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.323+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc63c00ab70 con 0x7fc6441539b0 2023-11-24T06:24:32.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7fc639ffb700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 0x7fc62806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc63c088b90 con 0x7fc6441539b0 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.323+0000 7f2487826700 1 Processor -- start 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.323+0000 7f2487826700 1 -- start start 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7f2487826700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2480164a90 0x7f248006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7f2487826700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f248006d4d0 0x7f248016bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7f2487826700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248006ced0 con 0x7f24800b3b50 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7f2487826700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248006da70 con 0x7f2480164a90 2023-11-24T06:24:32.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.324+0000 7f2487826700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24801634e0 con 0x7f248006d4d0 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f248006d4d0 0x7f248016bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f248006d4d0 0x7f248016bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:33908/0 (socket says 172.21.15.114:33908) 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 -- 172.21.15.114:0/2810151349 learned_addr learned my addr 172.21.15.114:0/2810151349 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 -- 172.21.15.114:0/2810151349 >> v2:172.21.15.114:3300/0 conn(0x7f2480164a90 msgr2=0x7f248006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 --2- 172.21.15.114:0/2810151349 >> v2:172.21.15.114:3300/0 conn(0x7f2480164a90 0x7f248006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 -- 172.21.15.114:0/2810151349 >> v1:172.21.15.172:6789/0 conn(0x7f24800b3b50 legacy=0x7f24800b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 -- 172.21.15.114:0/2810151349 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24801637c0 con 0x7f248006d4d0 2023-11-24T06:24:32.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.325+0000 7f2485dc3700 1 --2- 172.21.15.114:0/2810151349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248006d4d0 0x7f248016bee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2480070a50 tx=0x7f247800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9606f016eb1acde server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.328+0000 7fc6491d0700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 0x7fc62806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.328+0000 7fc6491d0700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 0x7fc62806d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc63400a010 tx=0x7fc634013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.328+0000 7f2477fff700 1 -- 172.21.15.114:0/2810151349 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247800e070 con 0x7f248006d4d0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.328+0000 7f2477fff700 1 -- 172.21.15.114:0/2810151349 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247800b9c0 con 0x7f248006d4d0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.328+0000 7f2477fff700 1 -- 172.21.15.114:0/2810151349 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247800bd70 con 0x7f248006d4d0 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.329+0000 7f2487826700 1 -- 172.21.15.114:0/2810151349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248006d4d0 msgr2=0x7f248016bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.329+0000 7f2487826700 1 --2- 172.21.15.114:0/2810151349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248006d4d0 0x7f248016bee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2480070a50 tx=0x7f247800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:32.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 -- 172.21.15.114:0/2810151349 shutdown_connections 2023-11-24T06:24:32.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 --2- 172.21.15.114:0/2810151349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248006d4d0 0x7f248016bee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 --2- 172.21.15.114:0/2810151349 >> v2:172.21.15.114:3300/0 conn(0x7f2480164a90 0x7f248006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 -- 172.21.15.114:0/2810151349 >> 172.21.15.114:0/2810151349 conn(0x7f24800cfa30 msgr2=0x7f24800cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 -- 172.21.15.114:0/2810151349 shutdown_connections 2023-11-24T06:24:32.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7f2487826700 1 -- 172.21.15.114:0/2810151349 wait complete. 2023-11-24T06:24:32.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.335+0000 7f2487826700 1 Processor -- start 2023-11-24T06:24:32.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.335+0000 7f2487826700 1 -- start start 2023-11-24T06:24:32.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24800b3b50 0x7f2480070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2480164a90 0x7f2480071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2480147ff0 con 0x7f24800b3b50 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24801482d0 con 0x7f2480164a90 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2487826700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24801485b0 con 0x7f248006d4d0 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:32802/0 (socket says 172.21.15.114:32802) 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 -- 172.21.15.114:0/2504596699 learned_addr learned my addr 172.21.15.114:0/2504596699 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f2484dc1700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24800b3b50 0x7f2480070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 -- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2480164a90 msgr2=0x7f2480071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2480164a90 0x7f2480071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 -- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24800b3b50 msgr2=0x7f2480070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24800b3b50 0x7f2480070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 -- 172.21.15.114:0/2504596699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009b30 con 0x7f248006d4d0 2023-11-24T06:24:32.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.337+0000 7f24855c2700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c4e0 tx=0x7f247c00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.338+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0052d0 con 0x7f248006d4d0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.338+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c005430 con 0x7f248006d4d0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.338+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2480148890 con 0x7f248006d4d0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.338+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2480147ca0 con 0x7f248006d4d0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.339+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0038c0 con 0x7f248006d4d0 2023-11-24T06:24:32.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.340+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2480148db0 con 0x7f248006d4d0 2023-11-24T06:24:32.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.341+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f247c003a20 con 0x7f248006d4d0 2023-11-24T06:24:32.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.342+0000 7f2475ffb700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 0x7f246c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.342+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f247c088bd0 con 0x7f248006d4d0 2023-11-24T06:24:32.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.343+0000 7f2484dc1700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 0x7f246c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.343+0000 7f2484dc1700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 0x7f246c06d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2480150180 tx=0x7f2470009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.333+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6cc055680 con 0x7fe6d414ea30 2023-11-24T06:24:32.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.344+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6440b39f0 con 0x7fc6441539b0 2023-11-24T06:24:32.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.351+0000 7fb0242ac700 1 Processor -- start 2023-11-24T06:24:32.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.351+0000 7fb0242ac700 1 -- start start 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.352+0000 7fb0242ac700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb01c072bb0 0x7fb01c16abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.352+0000 7fb0242ac700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c16b130 con 0x7fb01c0b3b50 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.352+0000 7fb0242ac700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c16f5e0 con 0x7fb01c071d10 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.352+0000 7fb0242ac700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c06ce30 con 0x7fb01c072bb0 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb021847700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb01c0b3b50 0x7fb01c16e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53976/0 (socket says 172.21.15.114:53976) 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb021847700 1 -- 172.21.15.114:0/952353361 learned_addr learned my addr 172.21.15.114:0/952353361 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:32.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb022048700 1 --2- 172.21.15.114:0/952353361 >> v2:172.21.15.172:3300/0 conn(0x7fb01c072bb0 0x7fb01c16abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb022048700 1 -- 172.21.15.114:0/952353361 >> v1:172.21.15.172:6789/0 conn(0x7fb01c071d10 legacy=0x7fb01c16eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb022048700 1 -- 172.21.15.114:0/952353361 >> v1:172.21.15.114:6789/0 conn(0x7fb01c0b3b50 legacy=0x7fb01c16e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb022048700 1 -- 172.21.15.114:0/952353361 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c06d110 con 0x7fb01c072bb0 2023-11-24T06:24:32.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.353+0000 7fb022048700 1 --2- 172.21.15.114:0/952353361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c072bb0 0x7fb01c16abf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb014009d90 tx=0x7fb01400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c58166264eb90c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.356+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f247c053df0 con 0x7f248006d4d0 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.359+0000 7fb021046700 1 -- 172.21.15.114:0/952353361 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400e070 con 0x7fb01c072bb0 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.359+0000 7fb021046700 1 -- 172.21.15.114:0/952353361 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01400bad0 con 0x7fb01c072bb0 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.359+0000 7fb021046700 1 -- 172.21.15.114:0/952353361 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400be80 con 0x7fb01c072bb0 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.360+0000 7fb0242ac700 1 -- 172.21.15.114:0/952353361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c072bb0 msgr2=0x7fb01c16abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.360+0000 7fb0242ac700 1 --2- 172.21.15.114:0/952353361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c072bb0 0x7fb01c16abf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb014009d90 tx=0x7fb01400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.360+0000 7fb0242ac700 1 -- 172.21.15.114:0/952353361 shutdown_connections 2023-11-24T06:24:32.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.360+0000 7fb0242ac700 1 --2- 172.21.15.114:0/952353361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c072bb0 0x7fb01c16abf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.360+0000 7fb0242ac700 1 -- 172.21.15.114:0/952353361 >> 172.21.15.114:0/952353361 conn(0x7fb01c0cfa30 msgr2=0x7fb01c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.370+0000 7fb0242ac700 1 -- 172.21.15.114:0/952353361 shutdown_connections 2023-11-24T06:24:32.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.370+0000 7fb0242ac700 1 -- 172.21.15.114:0/952353361 wait complete. 2023-11-24T06:24:32.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.372+0000 7fb0242ac700 1 Processor -- start 2023-11-24T06:24:32.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.372+0000 7fb0242ac700 1 -- start start 2023-11-24T06:24:32.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c071d10 0x7fb01c0aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0b3b50 0x7fb01c0ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0ac7b0 con 0x7fb01c0acc40 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0abba0 con 0x7fb01c071d10 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.373+0000 7fb0242ac700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0abe80 con 0x7fb01c0b3b50 2023-11-24T06:24:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38276/0 (socket says 172.21.15.114:38276) 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 -- 172.21.15.114:0/1392204765 learned_addr learned my addr 172.21.15.114:0/1392204765 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 -- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0b3b50 msgr2=0x7fb01c0ab370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0b3b50 0x7fb01c0ab370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 -- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c071d10 msgr2=0x7fb01c0aadb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c071d10 0x7fb01c0aadb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.374+0000 7fb022849700 1 -- 172.21.15.114:0/1392204765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014009a70 con 0x7fb01c0acc40 2023-11-24T06:24:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb022849700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c340 tx=0x7fb00c00c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0052d0 con 0x7fb01c0acc40 2023-11-24T06:24:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c005450 con 0x7fb01c0acc40 2023-11-24T06:24:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0038c0 con 0x7fb01c0acc40 2023-11-24T06:24:32.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0ac160 con 0x7fb01c0acc40 2023-11-24T06:24:32.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.375+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0ab8b0 con 0x7fb01c0acc40 2023-11-24T06:24:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.379+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb00c003a20 con 0x7fb01c0acc40 2023-11-24T06:24:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.379+0000 7fb012ffd700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 0x7fb00806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.379+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb00c088880 con 0x7fb01c0acc40 2023-11-24T06:24:32.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.385+0000 7fb022048700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 0x7fb00806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.386+0000 7fb022048700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 0x7fb00806d5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b370 tx=0x7fb014011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.386+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0040052e0 con 0x7fb01c0acc40 2023-11-24T06:24:32.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.396+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb00c0539a0 con 0x7fb01c0acc40 2023-11-24T06:24:32.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.602+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd3a80751e0 con 0x7fd3a8201330 2023-11-24T06:24:32.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.602+0000 7fd39dffb700 1 -- 172.21.15.114:0/217903363 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3a00583c0 con 0x7fd3a8201330 2023-11-24T06:24:32.609 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.606+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 msgr2=0x7fd38c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.606+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 0x7fd38c06d490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a4007a50 comp rx=0 tx=0).stop 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.606+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 msgr2=0x7fd3a8168960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.606+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0007e30 tx=0x7fd3a000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 shutdown_connections 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd38c06afd0 0x7fd38c06d490 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3a8201330 0x7fd3a8168960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3a80b4260 0x7fd3a8162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 --2- 172.21.15.114:0/217903363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3a8162b50 0x7fd3a8200df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 >> 172.21.15.114:0/217903363 conn(0x7fd3a80cfa30 msgr2=0x7fd3a81630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 shutdown_connections 2023-11-24T06:24:32.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.607+0000 7fd3af42c700 1 -- 172.21.15.114:0/217903363 wait complete. 2023-11-24T06:24:32.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-11-24T06:24:32.629 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.652+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2720071f00 con 0x7f27200b0f80 2023-11-24T06:24:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.652+0000 7f2707fff700 1 -- 172.21.15.114:0/121834393 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2710015030 con 0x7f27200b0f80 2023-11-24T06:24:32.658 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 msgr2=0x7f270806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 0x7f270806d580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2714006b20 tx=0x7f2714006ab0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 msgr2=0x7f2720159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f271000b530 tx=0x7f271000b840 comp rx=0 tx=0).stop 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 shutdown_connections 2023-11-24T06:24:32.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f270806b0c0 0x7f270806d580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272015ee90 0x7f27200b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27200b0ba0 0x7f2720200c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 --2- 172.21.15.114:0/121834393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27200b0f80 0x7f2720159410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 >> 172.21.15.114:0/121834393 conn(0x7f27200b57b0 msgr2=0x7f27200cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 shutdown_connections 2023-11-24T06:24:32.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.659+0000 7f2725a85700 1 -- 172.21.15.114:0/121834393 wait complete. 2023-11-24T06:24:32.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.671+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efd38005860 con 0x7efd580aee30 2023-11-24T06:24:32.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-11-24T06:24:32.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.672+0000 7efd54ff9700 1 -- 172.21.15.114:0/1491792813 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd4c015090 con 0x7efd580aee30 2023-11-24T06:24:32.676 INFO:teuthology.orchestra.run.smithi114.stdout:55834574859 2023-11-24T06:24:32.676 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.675+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 msgr2=0x7efd4006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.675+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 0x7efd4006d580 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd48014040 tx=0x7efd48009000 comp rx=0 tx=0).stop 2023-11-24T06:24:32.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.675+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 msgr2=0x7efd580ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.675+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 0x7efd580ce950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006fd0 tx=0x7efd4c00b730 comp rx=0 tx=0).stop 2023-11-24T06:24:32.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 shutdown_connections 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efd4006b0c0 0x7efd4006d580 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd580cee90 0x7efd58169380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd580b4260 0x7efd580ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 --2- 172.21.15.114:0/1491792813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd580aee30 0x7efd580ce950 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 >> 172.21.15.114:0/1491792813 conn(0x7efd580cfa30 msgr2=0x7efd580cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 shutdown_connections 2023-11-24T06:24:32.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.676+0000 7efd5e4eb700 1 -- 172.21.15.114:0/1491792813 wait complete. 2023-11-24T06:24:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.688+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8c180cbc50 con 0x7f8c00005000 2023-11-24T06:24:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.688+0000 7f8bfffff700 1 -- 172.21.15.114:0/3717480769 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c180cbc50 con 0x7f8c00005000 2023-11-24T06:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.692+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 msgr2=0x7f8c0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.692+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 0x7f8c0406d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c18156d50 tx=0x7f8c0c013000 comp rx=0 tx=0).stop 2023-11-24T06:24:32.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.692+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 msgr2=0x7f8c1808a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.692+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8c10002150 tx=0x7f8c1000a340 comp rx=0 tx=0).stop 2023-11-24T06:24:32.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2023-11-24T06:24:32.696 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 shutdown_connections 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c0406aed0 0x7f8c0406d390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c181652b0 0x7f8c18165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c00005000 0x7f8c1808a9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 --2- 172.21.15.114:0/3717480769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c18166910 0x7f8c18166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 >> 172.21.15.114:0/3717480769 conn(0x7f8c180cfa30 msgr2=0x7f8c180b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 shutdown_connections 2023-11-24T06:24:32.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.693+0000 7f8c1ddce700 1 -- 172.21.15.114:0/3717480769 wait complete. 2023-11-24T06:24:32.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-11-24T06:24:32.709 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.723+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc6440b4390 con 0x7fc6441539b0 2023-11-24T06:24:32.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.723+0000 7fc639ffb700 1 -- 172.21.15.114:0/592730133 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6440b4390 con 0x7fc6441539b0 2023-11-24T06:24:32.726 INFO:teuthology.orchestra.run.smithi114.stdout:55834574858 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 msgr2=0x7fc62806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 0x7fc62806d3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc63400a010 tx=0x7fc634013000 comp rx=0 tx=0).stop 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 msgr2=0x7fc6440ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ce310 tx=0x7fc63c00ce00 comp rx=0 tx=0).stop 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 shutdown_connections 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc62806af20 0x7fc62806d3e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6441539b0 0x7fc6440ad270 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6441535d0 0x7fc6440b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 --2- 172.21.15.114:0/592730133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6440b4670 0x7fc6440b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 >> 172.21.15.114:0/592730133 conn(0x7fc6440b57b0 msgr2=0x7fc644061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 shutdown_connections 2023-11-24T06:24:32.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.725+0000 7fc64b434700 1 -- 172.21.15.114:0/592730133 wait complete. 2023-11-24T06:24:32.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.727+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24801486f0 con 0x7f248006d4d0 2023-11-24T06:24:32.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.728+0000 7f2475ffb700 1 -- 172.21.15.114:0/2504596699 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24801486f0 con 0x7f248006d4d0 2023-11-24T06:24:32.731 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 msgr2=0x7f246c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 0x7f246c06d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2480150180 tx=0x7f2470009110 comp rx=0 tx=0).stop 2023-11-24T06:24:32.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 msgr2=0x7f248014b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f247c00c4e0 tx=0x7f247c00c7f0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 shutdown_connections 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f246c06b0b0 0x7f246c06d570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2480164a90 0x7f2480071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f248006d4d0 0x7f248014b630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 --2- 172.21.15.114:0/2504596699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24800b3b50 0x7f2480070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 >> 172.21.15.114:0/2504596699 conn(0x7f24800cfa30 msgr2=0x7f24801502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe6bc005590 con 0x7fe6d414ea30 2023-11-24T06:24:32.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 shutdown_connections 2023-11-24T06:24:32.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.730+0000 7f2487826700 1 -- 172.21.15.114:0/2504596699 wait complete. 2023-11-24T06:24:32.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.731+0000 7fe6ca7fc700 1 -- 172.21.15.114:0/3638137032 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe6cc017070 con 0x7fe6d414ea30 2023-11-24T06:24:32.735 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.735+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 msgr2=0x7fe6b808f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.735+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 0x7fe6b808f470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40737a0 tx=0x7fe6c4009aa0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.735+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 msgr2=0x7fe6d41662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.735+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 0x7fe6d41662f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc014000 tx=0x7fe6cc001650 comp rx=0 tx=0).stop 2023-11-24T06:24:32.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 shutdown_connections 2023-11-24T06:24:32.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe6b808cfb0 0x7fe6b808f470 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d414ea30 0x7fe6d41662f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6c0006320 0x7fe6d414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 --2- 172.21.15.114:0/3638137032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6d4150030 0x7fe6d414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 >> 172.21.15.114:0/3638137032 conn(0x7fe6d40cfa30 msgr2=0x7fe6d40cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 shutdown_connections 2023-11-24T06:24:32.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.737+0000 7fe6dbaf1700 1 -- 172.21.15.114:0/3638137032 wait complete. 2023-11-24T06:24:32.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2023-11-24T06:24:32.745 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-11-24T06:24:32.747 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-11-24T06:24:32.759 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.773+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb004005590 con 0x7fb01c0acc40 2023-11-24T06:24:32.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.773+0000 7fb012ffd700 1 -- 172.21.15.114:0/1392204765 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb00c017070 con 0x7fb01c0acc40 2023-11-24T06:24:32.777 INFO:teuthology.orchestra.run.smithi114.stdout:55834574857 2023-11-24T06:24:32.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 msgr2=0x7fb00806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 0x7fb00806d5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b370 tx=0x7fb014011000 comp rx=0 tx=0).stop 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 msgr2=0x7fb01c15e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c340 tx=0x7fb00c00c700 comp rx=0 tx=0).stop 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 shutdown_connections 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb00806b130 0x7fb00806d5f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb01c0acc40 0x7fb01c15e3a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb01c071d10 0x7fb01c0aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 --2- 172.21.15.114:0/1392204765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb01c0b3b50 0x7fb01c0ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 >> 172.21.15.114:0/1392204765 conn(0x7fb01c0cfa30 msgr2=0x7fb01c16e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 shutdown_connections 2023-11-24T06:24:32.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.775+0000 7fb0242ac700 1 -- 172.21.15.114:0/1392204765 wait complete. 2023-11-24T06:24:32.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-24T06:24:32.791 DEBUG:teuthology.parallel:result is None 2023-11-24T06:24:32.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:24:32.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:32.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.949+0000 7f319c1ea700 1 Processor -- start 2023-11-24T06:24:32.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.949+0000 7f319c1ea700 1 -- start start 2023-11-24T06:24:32.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f319415bdd0 0x7f31941604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31940b3370 0x7f31940b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f31940d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940b2f50 con 0x7f31940b3370 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940d2660 con 0x7f319415bdd0 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319c1ea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940b2030 con 0x7f31940d1c70 2023-11-24T06:24:32.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31940b3370 0x7f31940b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31940b3370 0x7f31940b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38280/0 (socket says 172.21.15.114:38280) 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 -- 172.21.15.114:0/3371516664 learned_addr learned my addr 172.21.15.114:0/3371516664 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f319a787700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f31940d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199f86700 1 --2- 172.21.15.114:0/3371516664 >> v2:172.21.15.172:3300/0 conn(0x7f319415bdd0 0x7f31941604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 -- 172.21.15.114:0/3371516664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 msgr2=0x7f31940d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f31940d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 -- 172.21.15.114:0/3371516664 >> v2:172.21.15.172:3300/0 conn(0x7f319415bdd0 msgr2=0x7f31941604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:32.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 --2- 172.21.15.114:0/3371516664 >> v2:172.21.15.172:3300/0 conn(0x7f319415bdd0 0x7f31941604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 -- 172.21.15.114:0/3371516664 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940b22d0 con 0x7f31940b3370 2023-11-24T06:24:32.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3199785700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31940b3370 0x7f31940b1af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f319406ec10 tx=0x7f3190009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242a972c3fad691 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3198f84700 1 -- 172.21.15.114:0/3371516664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000b870 con 0x7f31940b3370 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3198f84700 1 -- 172.21.15.114:0/3371516664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000b9d0 con 0x7f31940b3370 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.950+0000 7f3198f84700 1 -- 172.21.15.114:0/3371516664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000bd60 con 0x7f31940b3370 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 -- 172.21.15.114:0/3371516664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31940b3370 msgr2=0x7f31940b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31940b3370 0x7f31940b1af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f319406ec10 tx=0x7f3190009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 -- 172.21.15.114:0/3371516664 shutdown_connections 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f31940d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 --2- 172.21.15.114:0/3371516664 >> v2:172.21.15.172:3300/0 conn(0x7f319415bdd0 0x7f31941604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 --2- 172.21.15.114:0/3371516664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31940b3370 0x7f31940b1af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 -- 172.21.15.114:0/3371516664 >> 172.21.15.114:0/3371516664 conn(0x7f31940731a0 msgr2=0x7f319409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 -- 172.21.15.114:0/3371516664 shutdown_connections 2023-11-24T06:24:32.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.951+0000 7f319c1ea700 1 -- 172.21.15.114:0/3371516664 wait complete. 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.952+0000 7f319c1ea700 1 Processor -- start 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.952+0000 7f319c1ea700 1 -- start start 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31940b3370 0x7f31940c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f319414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940b4fa0 con 0x7f319415bdd0 2023-11-24T06:24:32.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c5fb0 con 0x7f31940b3370 2023-11-24T06:24:32.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319c1ea700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c6290 con 0x7f31940d1c70 2023-11-24T06:24:32.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319a787700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.953+0000 7f319a787700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38290/0 (socket says 172.21.15.114:38290) 2023-11-24T06:24:32.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 -- 172.21.15.114:0/2357847781 learned_addr learned my addr 172.21.15.114:0/2357847781 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:32.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f3199f86700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31940b3370 0x7f31940c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f3199785700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f319414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 -- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 msgr2=0x7f319414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f319414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 -- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31940b3370 msgr2=0x7f31940c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31940b3370 0x7f31940c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 -- 172.21.15.114:0/2357847781 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3190009550 con 0x7f319415bdd0 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.954+0000 7f319a787700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ccf0 tx=0x7f318c00a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.955+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c00ad60 con 0x7f319415bdd0 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.955+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c003a20 con 0x7f319415bdd0 2023-11-24T06:24:32.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.955+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940c6570 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.956+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003db0 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.956+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940c5ca0 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.957+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31940c6a90 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.957+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f318c003fd0 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.957+0000 7f318a7fc700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 0x7f317806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.957+0000 7f3199f86700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 0x7f317806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.957+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f318c012070 con 0x7f319415bdd0 2023-11-24T06:24:32.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.958+0000 7f3199f86700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 0x7f317806d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3184006fd0 tx=0x7f3184008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:32.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:32.962+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f318c054d30 con 0x7f319415bdd0 2023-11-24T06:24:33.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.148+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31940b2b70 con 0x7f317806aed0 2023-11-24T06:24:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.155+0000 7f318a7fc700 1 -- 172.21.15.114:0/2357847781 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264607 (secure 0 0 0) 0x7f318400c040 con 0x7f317806aed0 2023-11-24T06:24:33.162 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.158+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 msgr2=0x7f317806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.158+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 0x7f317806d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3184006fd0 tx=0x7f3184008040 comp rx=0 tx=0).stop 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.158+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 msgr2=0x7f319415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.158+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f318c00ccf0 tx=0x7f318c00a290 comp rx=0 tx=0).stop 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 shutdown_connections 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f317806aed0 0x7f317806d390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f319415bdd0 0x7f319415a170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31940b3370 0x7f31940c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 --2- 172.21.15.114:0/2357847781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31940d1c70 0x7f319414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 >> 172.21.15.114:0/2357847781 conn(0x7f31940731a0 msgr2=0x7f31940b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 shutdown_connections 2023-11-24T06:24:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.159+0000 7f319c1ea700 1 -- 172.21.15.114:0/2357847781 wait complete. 2023-11-24T06:24:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:33.174 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-24T06:24:31.710230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219108,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755612,"statfs":{"total":773094113280,"available":772869746688,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.858893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854842+0000","last_change":"2023-11-24T06:24:19.869889+0000","last_active":"2023-11-24T06:24:20.854842+0000","last_peered":"2023-11-24T06:24:20.854842+0000","last_clean":"2023-11-24T06:24:20.854842+0000","last_became_active":"2023-11-24T06:24:19.869608+0000","last_became_peered":"2023-11-24T06:24:19.869608+0000","last_unstale":"2023-11-24T06:24:20.854842+0000","last_undegraded":"2023-11-24T06:24:20.854842+0000","last_fullsized":"2023-11-24T06:24:20.854842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:48.551881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853246+0000","last_change":"2023-11-24T06:24:19.872814+0000","last_active":"2023-11-24T06:24:20.853246+0000","last_peered":"2023-11-24T06:24:20.853246+0000","last_clean":"2023-11-24T06:24:20.853246+0000","last_became_active":"2023-11-24T06:24:19.872430+0000","last_became_peered":"2023-11-24T06:24:19.872430+0000","last_unstale":"2023-11-24T06:24:20.853246+0000","last_undegraded":"2023-11-24T06:24:20.853246+0000","last_fullsized":"2023-11-24T06:24:20.853246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979460+0000","last_change":"2023-11-24T06:24:19.869057+0000","last_active":"2023-11-24T06:24:20.979460+0000","last_peered":"2023-11-24T06:24:20.979460+0000","last_clean":"2023-11-24T06:24:20.979460+0000","last_became_active":"2023-11-24T06:24:19.868779+0000","last_became_peered":"2023-11-24T06:24:19.868779+0000","last_unstale":"2023-11-24T06:24:20.979460+0000","last_undegraded":"2023-11-24T06:24:20.979460+0000","last_fullsized":"2023-11-24T06:24:20.979460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054845+0000","last_change":"2023-11-24T06:24:19.875414+0000","last_active":"2023-11-24T06:24:21.054845+0000","last_peered":"2023-11-24T06:24:21.054845+0000","last_clean":"2023-11-24T06:24:21.054845+0000","last_became_active":"2023-11-24T06:24:19.875156+0000","last_became_peered":"2023-11-24T06:24:19.875156+0000","last_unstale":"2023-11-24T06:24:21.054845+0000","last_undegraded":"2023-11-24T06:24:21.054845+0000","last_fullsized":"2023-11-24T06:24:21.054845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854274+0000","last_change":"2023-11-24T06:24:19.862612+0000","last_active":"2023-11-24T06:24:20.854274+0000","last_peered":"2023-11-24T06:24:20.854274+0000","last_clean":"2023-11-24T06:24:20.854274+0000","last_became_active":"2023-11-24T06:24:19.862302+0000","last_became_peered":"2023-11-24T06:24:19.862302+0000","last_unstale":"2023-11-24T06:24:20.854274+0000","last_undegraded":"2023-11-24T06:24:20.854274+0000","last_fullsized":"2023-11-24T06:24:20.854274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054394+0000","last_change":"2023-11-24T06:24:19.871642+0000","last_active":"2023-11-24T06:24:21.054394+0000","last_peered":"2023-11-24T06:24:21.054394+0000","last_clean":"2023-11-24T06:24:21.054394+0000","last_became_active":"2023-11-24T06:24:19.871363+0000","last_became_peered":"2023-11-24T06:24:19.871363+0000","last_unstale":"2023-11-24T06:24:21.054394+0000","last_undegraded":"2023-11-24T06:24:21.054394+0000","last_fullsized":"2023-11-24T06:24:21.054394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854102+0000","last_change":"2023-11-24T06:24:19.865691+0000","last_active":"2023-11-24T06:24:20.854102+0000","last_peered":"2023-11-24T06:24:20.854102+0000","last_clean":"2023-11-24T06:24:20.854102+0000","last_became_active":"2023-11-24T06:24:19.865409+0000","last_became_peered":"2023-11-24T06:24:19.865409+0000","last_unstale":"2023-11-24T06:24:20.854102+0000","last_undegraded":"2023-11-24T06:24:20.854102+0000","last_fullsized":"2023-11-24T06:24:20.854102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904022+0000","last_change":"2023-11-24T06:24:19.866800+0000","last_active":"2023-11-24T06:24:20.904022+0000","last_peered":"2023-11-24T06:24:20.904022+0000","last_clean":"2023-11-24T06:24:20.904022+0000","last_became_active":"2023-11-24T06:24:19.866162+0000","last_became_peered":"2023-11-24T06:24:19.866162+0000","last_unstale":"2023-11-24T06:24:20.904022+0000","last_undegraded":"2023-11-24T06:24:20.904022+0000","last_fullsized":"2023-11-24T06:24:20.904022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854764+0000","last_change":"2023-11-24T06:24:19.867736+0000","last_active":"2023-11-24T06:24:20.854764+0000","last_peered":"2023-11-24T06:24:20.854764+0000","last_clean":"2023-11-24T06:24:20.854764+0000","last_became_active":"2023-11-24T06:24:19.867377+0000","last_became_peered":"2023-11-24T06:24:19.867377+0000","last_unstale":"2023-11-24T06:24:20.854764+0000","last_undegraded":"2023-11-24T06:24:20.854764+0000","last_fullsized":"2023-11-24T06:24:20.854764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553520+0000","last_change":"2023-11-24T06:24:19.870504+0000","last_active":"2023-11-24T06:24:21.553520+0000","last_peered":"2023-11-24T06:24:21.553520+0000","last_clean":"2023-11-24T06:24:21.553520+0000","last_became_active":"2023-11-24T06:24:19.870143+0000","last_became_peered":"2023-11-24T06:24:19.870143+0000","last_unstale":"2023-11-24T06:24:21.553520+0000","last_undegraded":"2023-11-24T06:24:21.553520+0000","last_fullsized":"2023-11-24T06:24:21.553520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553151+0000","last_change":"2023-11-24T06:24:19.873638+0000","last_active":"2023-11-24T06:24:21.553151+0000","last_peered":"2023-11-24T06:24:21.553151+0000","last_clean":"2023-11-24T06:24:21.553151+0000","last_became_active":"2023-11-24T06:24:19.871551+0000","last_became_peered":"2023-11-24T06:24:19.871551+0000","last_unstale":"2023-11-24T06:24:21.553151+0000","last_undegraded":"2023-11-24T06:24:21.553151+0000","last_fullsized":"2023-11-24T06:24:21.553151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855889+0000","last_change":"2023-11-24T06:24:19.867808+0000","last_active":"2023-11-24T06:24:20.855889+0000","last_peered":"2023-11-24T06:24:20.855889+0000","last_clean":"2023-11-24T06:24:20.855889+0000","last_became_active":"2023-11-24T06:24:19.867487+0000","last_became_peered":"2023-11-24T06:24:19.867487+0000","last_unstale":"2023-11-24T06:24:20.855889+0000","last_undegraded":"2023-11-24T06:24:20.855889+0000","last_fullsized":"2023-11-24T06:24:20.855889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904066+0000","last_change":"2023-11-24T06:24:19.870784+0000","last_active":"2023-11-24T06:24:20.904066+0000","last_peered":"2023-11-24T06:24:20.904066+0000","last_clean":"2023-11-24T06:24:20.904066+0000","last_became_active":"2023-11-24T06:24:19.870374+0000","last_became_peered":"2023-11-24T06:24:19.870374+0000","last_unstale":"2023-11-24T06:24:20.904066+0000","last_undegraded":"2023-11-24T06:24:20.904066+0000","last_fullsized":"2023-11-24T06:24:20.904066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979266+0000","last_change":"2023-11-24T06:24:19.865517+0000","last_active":"2023-11-24T06:24:20.979266+0000","last_peered":"2023-11-24T06:24:20.979266+0000","last_clean":"2023-11-24T06:24:20.979266+0000","last_became_active":"2023-11-24T06:24:19.865195+0000","last_became_peered":"2023-11-24T06:24:19.865195+0000","last_unstale":"2023-11-24T06:24:20.979266+0000","last_undegraded":"2023-11-24T06:24:20.979266+0000","last_fullsized":"2023-11-24T06:24:20.979266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852902+0000","last_change":"2023-11-24T06:24:19.867596+0000","last_active":"2023-11-24T06:24:20.852902+0000","last_peered":"2023-11-24T06:24:20.852902+0000","last_clean":"2023-11-24T06:24:20.852902+0000","last_became_active":"2023-11-24T06:24:19.867302+0000","last_became_peered":"2023-11-24T06:24:19.867302+0000","last_unstale":"2023-11-24T06:24:20.852902+0000","last_undegraded":"2023-11-24T06:24:20.852902+0000","last_fullsized":"2023-11-24T06:24:20.852902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054302+0000","last_change":"2023-11-24T06:24:19.871929+0000","last_active":"2023-11-24T06:24:21.054302+0000","last_peered":"2023-11-24T06:24:21.054302+0000","last_clean":"2023-11-24T06:24:21.054302+0000","last_became_active":"2023-11-24T06:24:19.870555+0000","last_became_peered":"2023-11-24T06:24:19.870555+0000","last_unstale":"2023-11-24T06:24:21.054302+0000","last_undegraded":"2023-11-24T06:24:21.054302+0000","last_fullsized":"2023-11-24T06:24:21.054302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054970+0000","last_change":"2023-11-24T06:24:19.871912+0000","last_active":"2023-11-24T06:24:21.054970+0000","last_peered":"2023-11-24T06:24:21.054970+0000","last_clean":"2023-11-24T06:24:21.054970+0000","last_became_active":"2023-11-24T06:24:19.871473+0000","last_became_peered":"2023-11-24T06:24:19.871473+0000","last_unstale":"2023-11-24T06:24:21.054970+0000","last_undegraded":"2023-11-24T06:24:21.054970+0000","last_fullsized":"2023-11-24T06:24:21.054970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553079+0000","last_change":"2023-11-24T06:24:19.873774+0000","last_active":"2023-11-24T06:24:21.553079+0000","last_peered":"2023-11-24T06:24:21.553079+0000","last_clean":"2023-11-24T06:24:21.553079+0000","last_became_active":"2023-11-24T06:24:19.872880+0000","last_became_peered":"2023-11-24T06:24:19.872880+0000","last_unstale":"2023-11-24T06:24:21.553079+0000","last_undegraded":"2023-11-24T06:24:21.553079+0000","last_fullsized":"2023-11-24T06:24:21.553079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854750+0000","last_change":"2023-11-24T06:24:19.863508+0000","last_active":"2023-11-24T06:24:20.854750+0000","last_peered":"2023-11-24T06:24:20.854750+0000","last_clean":"2023-11-24T06:24:20.854750+0000","last_became_active":"2023-11-24T06:24:19.863100+0000","last_became_peered":"2023-11-24T06:24:19.863100+0000","last_unstale":"2023-11-24T06:24:20.854750+0000","last_undegraded":"2023-11-24T06:24:20.854750+0000","last_fullsized":"2023-11-24T06:24:20.854750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.892967+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:19.892967+0000","last_peered":"2023-11-24T06:24:19.892967+0000","last_clean":"2023-11-24T06:24:19.892967+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:19.892967+0000","last_undegraded":"2023-11-24T06:24:19.892967+0000","last_fullsized":"2023-11-24T06:24:19.892967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904290+0000","last_change":"2023-11-24T06:24:19.869582+0000","last_active":"2023-11-24T06:24:20.904290+0000","last_peered":"2023-11-24T06:24:20.904290+0000","last_clean":"2023-11-24T06:24:20.904290+0000","last_became_active":"2023-11-24T06:24:19.869283+0000","last_became_peered":"2023-11-24T06:24:19.869283+0000","last_unstale":"2023-11-24T06:24:20.904290+0000","last_undegraded":"2023-11-24T06:24:20.904290+0000","last_fullsized":"2023-11-24T06:24:20.904290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883379+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:19.883379+0000","last_peered":"2023-11-24T06:24:19.883379+0000","last_clean":"2023-11-24T06:24:19.883379+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:19.883379+0000","last_undegraded":"2023-11-24T06:24:19.883379+0000","last_fullsized":"2023-11-24T06:24:19.883379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979537+0000","last_change":"2023-11-24T06:24:19.869681+0000","last_active":"2023-11-24T06:24:20.979537+0000","last_peered":"2023-11-24T06:24:20.979537+0000","last_clean":"2023-11-24T06:24:20.979537+0000","last_became_active":"2023-11-24T06:24:19.869437+0000","last_became_peered":"2023-11-24T06:24:19.869437+0000","last_unstale":"2023-11-24T06:24:20.979537+0000","last_undegraded":"2023-11-24T06:24:20.979537+0000","last_fullsized":"2023-11-24T06:24:20.979537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879979+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:19.879979+0000","last_peered":"2023-11-24T06:24:19.879979+0000","last_clean":"2023-11-24T06:24:19.879979+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:19.879979+0000","last_undegraded":"2023-11-24T06:24:19.879979+0000","last_fullsized":"2023-11-24T06:24:19.879979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:29.087819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054525+0000","last_change":"2023-11-24T06:24:19.869851+0000","last_active":"2023-11-24T06:24:21.054525+0000","last_peered":"2023-11-24T06:24:21.054525+0000","last_clean":"2023-11-24T06:24:21.054525+0000","last_became_active":"2023-11-24T06:24:19.869550+0000","last_became_peered":"2023-11-24T06:24:19.869550+0000","last_unstale":"2023-11-24T06:24:21.054525+0000","last_undegraded":"2023-11-24T06:24:21.054525+0000","last_fullsized":"2023-11-24T06:24:21.054525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880299+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:19.880299+0000","last_peered":"2023-11-24T06:24:19.880299+0000","last_clean":"2023-11-24T06:24:19.880299+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:19.880299+0000","last_undegraded":"2023-11-24T06:24:19.880299+0000","last_fullsized":"2023-11-24T06:24:19.880299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054853+0000","last_change":"2023-11-24T06:24:19.870442+0000","last_active":"2023-11-24T06:24:21.054853+0000","last_peered":"2023-11-24T06:24:21.054853+0000","last_clean":"2023-11-24T06:24:21.054853+0000","last_became_active":"2023-11-24T06:24:19.870164+0000","last_became_peered":"2023-11-24T06:24:19.870164+0000","last_unstale":"2023-11-24T06:24:21.054853+0000","last_undegraded":"2023-11-24T06:24:21.054853+0000","last_fullsized":"2023-11-24T06:24:21.054853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883377+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:19.883377+0000","last_peered":"2023-11-24T06:24:19.883377+0000","last_clean":"2023-11-24T06:24:19.883377+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:19.883377+0000","last_undegraded":"2023-11-24T06:24:19.883377+0000","last_fullsized":"2023-11-24T06:24:19.883377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855381+0000","last_change":"2023-11-24T06:24:19.864863+0000","last_active":"2023-11-24T06:24:20.855381+0000","last_peered":"2023-11-24T06:24:20.855381+0000","last_clean":"2023-11-24T06:24:20.855381+0000","last_became_active":"2023-11-24T06:24:19.864621+0000","last_became_peered":"2023-11-24T06:24:19.864621+0000","last_unstale":"2023-11-24T06:24:20.855381+0000","last_undegraded":"2023-11-24T06:24:20.855381+0000","last_fullsized":"2023-11-24T06:24:20.855381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:51.049403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879363+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:19.879363+0000","last_peered":"2023-11-24T06:24:19.879363+0000","last_clean":"2023-11-24T06:24:19.879363+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:19.879363+0000","last_undegraded":"2023-11-24T06:24:19.879363+0000","last_fullsized":"2023-11-24T06:24:19.879363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979510+0000","last_change":"2023-11-24T06:24:19.866029+0000","last_active":"2023-11-24T06:24:20.979510+0000","last_peered":"2023-11-24T06:24:20.979510+0000","last_clean":"2023-11-24T06:24:20.979510+0000","last_became_active":"2023-11-24T06:24:19.865361+0000","last_became_peered":"2023-11-24T06:24:19.865361+0000","last_unstale":"2023-11-24T06:24:20.979510+0000","last_undegraded":"2023-11-24T06:24:20.979510+0000","last_fullsized":"2023-11-24T06:24:20.979510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852200+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:19.852200+0000","last_peered":"2023-11-24T06:24:19.852200+0000","last_clean":"2023-11-24T06:24:19.852200+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:19.852200+0000","last_undegraded":"2023-11-24T06:24:19.852200+0000","last_fullsized":"2023-11-24T06:24:19.852200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904187+0000","last_change":"2023-11-24T06:24:19.863134+0000","last_active":"2023-11-24T06:24:20.904187+0000","last_peered":"2023-11-24T06:24:20.904187+0000","last_clean":"2023-11-24T06:24:20.904187+0000","last_became_active":"2023-11-24T06:24:19.862679+0000","last_became_peered":"2023-11-24T06:24:19.862679+0000","last_unstale":"2023-11-24T06:24:20.904187+0000","last_undegraded":"2023-11-24T06:24:20.904187+0000","last_fullsized":"2023-11-24T06:24:20.904187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.862408+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:19.862408+0000","last_peered":"2023-11-24T06:24:19.862408+0000","last_clean":"2023-11-24T06:24:19.862408+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:19.862408+0000","last_undegraded":"2023-11-24T06:24:19.862408+0000","last_fullsized":"2023-11-24T06:24:19.862408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853050+0000","last_change":"2023-11-24T06:24:19.864439+0000","last_active":"2023-11-24T06:24:20.853050+0000","last_peered":"2023-11-24T06:24:20.853050+0000","last_clean":"2023-11-24T06:24:20.853050+0000","last_became_active":"2023-11-24T06:24:19.864128+0000","last_became_peered":"2023-11-24T06:24:19.864128+0000","last_unstale":"2023-11-24T06:24:20.853050+0000","last_undegraded":"2023-11-24T06:24:20.853050+0000","last_fullsized":"2023-11-24T06:24:20.853050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860879+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:19.860879+0000","last_peered":"2023-11-24T06:24:19.860879+0000","last_clean":"2023-11-24T06:24:19.860879+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:19.860879+0000","last_undegraded":"2023-11-24T06:24:19.860879+0000","last_fullsized":"2023-11-24T06:24:19.860879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054843+0000","last_change":"2023-11-24T06:24:19.874520+0000","last_active":"2023-11-24T06:24:21.054843+0000","last_peered":"2023-11-24T06:24:21.054843+0000","last_clean":"2023-11-24T06:24:21.054843+0000","last_became_active":"2023-11-24T06:24:19.872481+0000","last_became_peered":"2023-11-24T06:24:19.872481+0000","last_unstale":"2023-11-24T06:24:21.054843+0000","last_undegraded":"2023-11-24T06:24:21.054843+0000","last_fullsized":"2023-11-24T06:24:21.054843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.865095+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:19.865095+0000","last_peered":"2023-11-24T06:24:19.865095+0000","last_clean":"2023-11-24T06:24:19.865095+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:19.865095+0000","last_undegraded":"2023-11-24T06:24:19.865095+0000","last_fullsized":"2023-11-24T06:24:19.865095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:51.886938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979608+0000","last_change":"2023-11-24T06:24:19.867545+0000","last_active":"2023-11-24T06:24:20.979608+0000","last_peered":"2023-11-24T06:24:20.979608+0000","last_clean":"2023-11-24T06:24:20.979608+0000","last_became_active":"2023-11-24T06:24:19.867313+0000","last_became_peered":"2023-11-24T06:24:19.867313+0000","last_unstale":"2023-11-24T06:24:20.979608+0000","last_undegraded":"2023-11-24T06:24:20.979608+0000","last_fullsized":"2023-11-24T06:24:20.979608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885062+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:19.885062+0000","last_peered":"2023-11-24T06:24:19.885062+0000","last_clean":"2023-11-24T06:24:19.885062+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:19.885062+0000","last_undegraded":"2023-11-24T06:24:19.885062+0000","last_fullsized":"2023-11-24T06:24:19.885062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:08.350203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855876+0000","last_change":"2023-11-24T06:24:19.864361+0000","last_active":"2023-11-24T06:24:20.855876+0000","last_peered":"2023-11-24T06:24:20.855876+0000","last_clean":"2023-11-24T06:24:20.855876+0000","last_became_active":"2023-11-24T06:24:19.864077+0000","last_became_peered":"2023-11-24T06:24:19.864077+0000","last_unstale":"2023-11-24T06:24:20.855876+0000","last_undegraded":"2023-11-24T06:24:20.855876+0000","last_fullsized":"2023-11-24T06:24:20.855876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885718+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:19.885718+0000","last_peered":"2023-11-24T06:24:19.885718+0000","last_clean":"2023-11-24T06:24:19.885718+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:19.885718+0000","last_undegraded":"2023-11-24T06:24:19.885718+0000","last_fullsized":"2023-11-24T06:24:19.885718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854301+0000","last_change":"2023-11-24T06:24:19.858071+0000","last_active":"2023-11-24T06:24:20.854301+0000","last_peered":"2023-11-24T06:24:20.854301+0000","last_clean":"2023-11-24T06:24:20.854301+0000","last_became_active":"2023-11-24T06:24:19.857803+0000","last_became_peered":"2023-11-24T06:24:19.857803+0000","last_unstale":"2023-11-24T06:24:20.854301+0000","last_undegraded":"2023-11-24T06:24:20.854301+0000","last_fullsized":"2023-11-24T06:24:20.854301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.882906+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:19.882906+0000","last_peered":"2023-11-24T06:24:19.882906+0000","last_clean":"2023-11-24T06:24:19.882906+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:19.882906+0000","last_undegraded":"2023-11-24T06:24:19.882906+0000","last_fullsized":"2023-11-24T06:24:19.882906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979188+0000","last_change":"2023-11-24T06:24:19.870356+0000","last_active":"2023-11-24T06:24:20.979188+0000","last_peered":"2023-11-24T06:24:20.979188+0000","last_clean":"2023-11-24T06:24:20.979188+0000","last_became_active":"2023-11-24T06:24:19.870057+0000","last_became_peered":"2023-11-24T06:24:19.870057+0000","last_unstale":"2023-11-24T06:24:20.979188+0000","last_undegraded":"2023-11-24T06:24:20.979188+0000","last_fullsized":"2023-11-24T06:24:20.979188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:30.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864248+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:19.864248+0000","last_peered":"2023-11-24T06:24:19.864248+0000","last_clean":"2023-11-24T06:24:19.864248+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:19.864248+0000","last_undegraded":"2023-11-24T06:24:19.864248+0000","last_fullsized":"2023-11-24T06:24:19.864248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854429+0000","last_change":"2023-11-24T06:24:19.863989+0000","last_active":"2023-11-24T06:24:20.854429+0000","last_peered":"2023-11-24T06:24:20.854429+0000","last_clean":"2023-11-24T06:24:20.854429+0000","last_became_active":"2023-11-24T06:24:19.861502+0000","last_became_peered":"2023-11-24T06:24:19.861502+0000","last_unstale":"2023-11-24T06:24:20.854429+0000","last_undegraded":"2023-11-24T06:24:20.854429+0000","last_fullsized":"2023-11-24T06:24:20.854429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860367+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:19.860367+0000","last_peered":"2023-11-24T06:24:19.860367+0000","last_clean":"2023-11-24T06:24:19.860367+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:19.860367+0000","last_undegraded":"2023-11-24T06:24:19.860367+0000","last_fullsized":"2023-11-24T06:24:19.860367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553322+0000","last_change":"2023-11-24T06:24:19.873616+0000","last_active":"2023-11-24T06:24:21.553322+0000","last_peered":"2023-11-24T06:24:21.553322+0000","last_clean":"2023-11-24T06:24:21.553322+0000","last_became_active":"2023-11-24T06:24:19.873308+0000","last_became_peered":"2023-11-24T06:24:19.873308+0000","last_unstale":"2023-11-24T06:24:21.553322+0000","last_undegraded":"2023-11-24T06:24:21.553322+0000","last_fullsized":"2023-11-24T06:24:21.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.886102+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:19.886102+0000","last_peered":"2023-11-24T06:24:19.886102+0000","last_clean":"2023-11-24T06:24:19.886102+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:19.886102+0000","last_undegraded":"2023-11-24T06:24:19.886102+0000","last_fullsized":"2023-11-24T06:24:19.886102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854591+0000","last_change":"2023-11-24T06:24:19.865497+0000","last_active":"2023-11-24T06:24:20.854591+0000","last_peered":"2023-11-24T06:24:20.854591+0000","last_clean":"2023-11-24T06:24:20.854591+0000","last_became_active":"2023-11-24T06:24:19.863266+0000","last_became_peered":"2023-11-24T06:24:19.863266+0000","last_unstale":"2023-11-24T06:24:20.854591+0000","last_undegraded":"2023-11-24T06:24:20.854591+0000","last_fullsized":"2023-11-24T06:24:20.854591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883170+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:19.883170+0000","last_peered":"2023-11-24T06:24:19.883170+0000","last_clean":"2023-11-24T06:24:19.883170+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:19.883170+0000","last_undegraded":"2023-11-24T06:24:19.883170+0000","last_fullsized":"2023-11-24T06:24:19.883170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854478+0000","last_change":"2023-11-24T06:24:19.862689+0000","last_active":"2023-11-24T06:24:20.854478+0000","last_peered":"2023-11-24T06:24:20.854478+0000","last_clean":"2023-11-24T06:24:20.854478+0000","last_became_active":"2023-11-24T06:24:19.862254+0000","last_became_peered":"2023-11-24T06:24:19.862254+0000","last_unstale":"2023-11-24T06:24:20.854478+0000","last_undegraded":"2023-11-24T06:24:20.854478+0000","last_fullsized":"2023-11-24T06:24:20.854478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860268+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:19.860268+0000","last_peered":"2023-11-24T06:24:19.860268+0000","last_clean":"2023-11-24T06:24:19.860268+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:19.860268+0000","last_undegraded":"2023-11-24T06:24:19.860268+0000","last_fullsized":"2023-11-24T06:24:19.860268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054699+0000","last_change":"2023-11-24T06:24:19.876750+0000","last_active":"2023-11-24T06:24:21.054699+0000","last_peered":"2023-11-24T06:24:21.054699+0000","last_clean":"2023-11-24T06:24:21.054699+0000","last_became_active":"2023-11-24T06:24:19.876463+0000","last_became_peered":"2023-11-24T06:24:19.876463+0000","last_unstale":"2023-11-24T06:24:21.054699+0000","last_undegraded":"2023-11-24T06:24:21.054699+0000","last_fullsized":"2023-11-24T06:24:21.054699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864116+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:19.864116+0000","last_peered":"2023-11-24T06:24:19.864116+0000","last_clean":"2023-11-24T06:24:19.864116+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:19.864116+0000","last_undegraded":"2023-11-24T06:24:19.864116+0000","last_fullsized":"2023-11-24T06:24:19.864116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855678+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:20.855678+0000","last_peered":"2023-11-24T06:24:20.855678+0000","last_clean":"2023-11-24T06:24:20.855678+0000","last_became_active":"2023-11-24T06:24:19.866108+0000","last_became_peered":"2023-11-24T06:24:19.866108+0000","last_unstale":"2023-11-24T06:24:20.855678+0000","last_undegraded":"2023-11-24T06:24:20.855678+0000","last_fullsized":"2023-11-24T06:24:20.855678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:03.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852935+0000","last_change":"2023-11-24T06:24:19.865135+0000","last_active":"2023-11-24T06:24:20.852935+0000","last_peered":"2023-11-24T06:24:20.852935+0000","last_clean":"2023-11-24T06:24:20.852935+0000","last_became_active":"2023-11-24T06:24:19.864837+0000","last_became_peered":"2023-11-24T06:24:19.864837+0000","last_unstale":"2023-11-24T06:24:20.852935+0000","last_undegraded":"2023-11-24T06:24:20.852935+0000","last_fullsized":"2023-11-24T06:24:20.852935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857378+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:19.857378+0000","last_peered":"2023-11-24T06:24:19.857378+0000","last_clean":"2023-11-24T06:24:19.857378+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:19.857378+0000","last_undegraded":"2023-11-24T06:24:19.857378+0000","last_fullsized":"2023-11-24T06:24:19.857378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904182+0000","last_change":"2023-11-24T06:24:19.871367+0000","last_active":"2023-11-24T06:24:20.904182+0000","last_peered":"2023-11-24T06:24:20.904182+0000","last_clean":"2023-11-24T06:24:20.904182+0000","last_became_active":"2023-11-24T06:24:19.871058+0000","last_became_peered":"2023-11-24T06:24:19.871058+0000","last_unstale":"2023-11-24T06:24:20.904182+0000","last_undegraded":"2023-11-24T06:24:20.904182+0000","last_fullsized":"2023-11-24T06:24:20.904182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.858425+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:19.858425+0000","last_peered":"2023-11-24T06:24:19.858425+0000","last_clean":"2023-11-24T06:24:19.858425+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:19.858425+0000","last_undegraded":"2023-11-24T06:24:19.858425+0000","last_fullsized":"2023-11-24T06:24:19.858425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852316+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:19.852316+0000","last_peered":"2023-11-24T06:24:19.852316+0000","last_clean":"2023-11-24T06:24:19.852316+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:19.852316+0000","last_undegraded":"2023-11-24T06:24:19.852316+0000","last_fullsized":"2023-11-24T06:24:19.852316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979177+0000","last_change":"2023-11-24T06:24:19.864862+0000","last_active":"2023-11-24T06:24:20.979177+0000","last_peered":"2023-11-24T06:24:20.979177+0000","last_clean":"2023-11-24T06:24:20.979177+0000","last_became_active":"2023-11-24T06:24:19.863541+0000","last_became_peered":"2023-11-24T06:24:19.863541+0000","last_unstale":"2023-11-24T06:24:20.979177+0000","last_undegraded":"2023-11-24T06:24:20.979177+0000","last_fullsized":"2023-11-24T06:24:20.979177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852544+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:19.852544+0000","last_peered":"2023-11-24T06:24:19.852544+0000","last_clean":"2023-11-24T06:24:19.852544+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:19.852544+0000","last_undegraded":"2023-11-24T06:24:19.852544+0000","last_fullsized":"2023-11-24T06:24:19.852544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.853979+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:19.853979+0000","last_peered":"2023-11-24T06:24:19.853979+0000","last_clean":"2023-11-24T06:24:19.853979+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:19.853979+0000","last_undegraded":"2023-11-24T06:24:19.853979+0000","last_fullsized":"2023-11-24T06:24:19.853979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:48.736229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853074+0000","last_change":"2023-11-24T06:24:19.870498+0000","last_active":"2023-11-24T06:24:20.853074+0000","last_peered":"2023-11-24T06:24:20.853074+0000","last_clean":"2023-11-24T06:24:20.853074+0000","last_became_active":"2023-11-24T06:24:19.870199+0000","last_became_peered":"2023-11-24T06:24:19.870199+0000","last_unstale":"2023-11-24T06:24:20.853074+0000","last_undegraded":"2023-11-24T06:24:20.853074+0000","last_fullsized":"2023-11-24T06:24:20.853074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853198+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:20.853198+0000","last_peered":"2023-11-24T06:24:20.853198+0000","last_clean":"2023-11-24T06:24:20.853198+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:20.853198+0000","last_undegraded":"2023-11-24T06:24:20.853198+0000","last_fullsized":"2023-11-24T06:24:20.853198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904362+0000","last_change":"2023-11-24T06:24:19.869468+0000","last_active":"2023-11-24T06:24:20.904362+0000","last_peered":"2023-11-24T06:24:20.904362+0000","last_clean":"2023-11-24T06:24:20.904362+0000","last_became_active":"2023-11-24T06:24:19.869176+0000","last_became_peered":"2023-11-24T06:24:19.869176+0000","last_unstale":"2023-11-24T06:24:20.904362+0000","last_undegraded":"2023-11-24T06:24:20.904362+0000","last_fullsized":"2023-11-24T06:24:20.904362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880276+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:19.880276+0000","last_peered":"2023-11-24T06:24:19.880276+0000","last_clean":"2023-11-24T06:24:19.880276+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:19.880276+0000","last_undegraded":"2023-11-24T06:24:19.880276+0000","last_fullsized":"2023-11-24T06:24:19.880276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979489+0000","last_change":"2023-11-24T06:24:19.868306+0000","last_active":"2023-11-24T06:24:20.979489+0000","last_peered":"2023-11-24T06:24:20.979489+0000","last_clean":"2023-11-24T06:24:20.979489+0000","last_became_active":"2023-11-24T06:24:19.867148+0000","last_became_peered":"2023-11-24T06:24:19.867148+0000","last_unstale":"2023-11-24T06:24:20.979489+0000","last_undegraded":"2023-11-24T06:24:20.979489+0000","last_fullsized":"2023-11-24T06:24:20.979489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852405+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:19.852405+0000","last_peered":"2023-11-24T06:24:19.852405+0000","last_clean":"2023-11-24T06:24:19.852405+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:19.852405+0000","last_undegraded":"2023-11-24T06:24:19.852405+0000","last_fullsized":"2023-11-24T06:24:19.852405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855451+0000","last_change":"2023-11-24T06:24:19.862616+0000","last_active":"2023-11-24T06:24:20.855451+0000","last_peered":"2023-11-24T06:24:20.855451+0000","last_clean":"2023-11-24T06:24:20.855451+0000","last_became_active":"2023-11-24T06:24:19.862284+0000","last_became_peered":"2023-11-24T06:24:19.862284+0000","last_unstale":"2023-11-24T06:24:20.855451+0000","last_undegraded":"2023-11-24T06:24:20.855451+0000","last_fullsized":"2023-11-24T06:24:20.855451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:46.608575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859914+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:19.859914+0000","last_peered":"2023-11-24T06:24:19.859914+0000","last_clean":"2023-11-24T06:24:19.859914+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:19.859914+0000","last_undegraded":"2023-11-24T06:24:19.859914+0000","last_fullsized":"2023-11-24T06:24:19.859914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904102+0000","last_change":"2023-11-24T06:24:19.866929+0000","last_active":"2023-11-24T06:24:20.904102+0000","last_peered":"2023-11-24T06:24:20.904102+0000","last_clean":"2023-11-24T06:24:20.904102+0000","last_became_active":"2023-11-24T06:24:19.866522+0000","last_became_peered":"2023-11-24T06:24:19.866522+0000","last_unstale":"2023-11-24T06:24:20.904102+0000","last_undegraded":"2023-11-24T06:24:20.904102+0000","last_fullsized":"2023-11-24T06:24:20.904102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859599+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:19.859599+0000","last_peered":"2023-11-24T06:24:19.859599+0000","last_clean":"2023-11-24T06:24:19.859599+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:19.859599+0000","last_undegraded":"2023-11-24T06:24:19.859599+0000","last_fullsized":"2023-11-24T06:24:19.859599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904434+0000","last_change":"2023-11-24T06:24:19.870623+0000","last_active":"2023-11-24T06:24:20.904434+0000","last_peered":"2023-11-24T06:24:20.904434+0000","last_clean":"2023-11-24T06:24:20.904434+0000","last_became_active":"2023-11-24T06:24:19.869946+0000","last_became_peered":"2023-11-24T06:24:19.869946+0000","last_unstale":"2023-11-24T06:24:20.904434+0000","last_undegraded":"2023-11-24T06:24:20.904434+0000","last_fullsized":"2023-11-24T06:24:20.904434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879779+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:19.879779+0000","last_peered":"2023-11-24T06:24:19.879779+0000","last_clean":"2023-11-24T06:24:19.879779+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:19.879779+0000","last_undegraded":"2023-11-24T06:24:19.879779+0000","last_fullsized":"2023-11-24T06:24:19.879779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:08.503960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553085+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:21.553085+0000","last_peered":"2023-11-24T06:24:21.553085+0000","last_clean":"2023-11-24T06:24:21.553085+0000","last_became_active":"2023-11-24T06:24:19.866115+0000","last_became_peered":"2023-11-24T06:24:19.866115+0000","last_unstale":"2023-11-24T06:24:21.553085+0000","last_undegraded":"2023-11-24T06:24:21.553085+0000","last_fullsized":"2023-11-24T06:24:21.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857112+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:19.857112+0000","last_peered":"2023-11-24T06:24:19.857112+0000","last_clean":"2023-11-24T06:24:19.857112+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:19.857112+0000","last_undegraded":"2023-11-24T06:24:19.857112+0000","last_fullsized":"2023-11-24T06:24:19.857112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854430+0000","last_change":"2023-11-24T06:24:19.867560+0000","last_active":"2023-11-24T06:24:20.854430+0000","last_peered":"2023-11-24T06:24:20.854430+0000","last_clean":"2023-11-24T06:24:20.854430+0000","last_became_active":"2023-11-24T06:24:19.867277+0000","last_became_peered":"2023-11-24T06:24:19.867277+0000","last_unstale":"2023-11-24T06:24:20.854430+0000","last_undegraded":"2023-11-24T06:24:20.854430+0000","last_fullsized":"2023-11-24T06:24:20.854430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885158+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:19.885158+0000","last_peered":"2023-11-24T06:24:19.885158+0000","last_clean":"2023-11-24T06:24:19.885158+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:19.885158+0000","last_undegraded":"2023-11-24T06:24:19.885158+0000","last_fullsized":"2023-11-24T06:24:19.885158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054548+0000","last_change":"2023-11-24T06:24:19.875426+0000","last_active":"2023-11-24T06:24:21.054548+0000","last_peered":"2023-11-24T06:24:21.054548+0000","last_clean":"2023-11-24T06:24:21.054548+0000","last_became_active":"2023-11-24T06:24:19.875167+0000","last_became_peered":"2023-11-24T06:24:19.875167+0000","last_unstale":"2023-11-24T06:24:21.054548+0000","last_undegraded":"2023-11-24T06:24:21.054548+0000","last_fullsized":"2023-11-24T06:24:21.054548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.884010+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:19.884010+0000","last_peered":"2023-11-24T06:24:19.884010+0000","last_clean":"2023-11-24T06:24:19.884010+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:19.884010+0000","last_undegraded":"2023-11-24T06:24:19.884010+0000","last_fullsized":"2023-11-24T06:24:19.884010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:50.045740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854899+0000","last_change":"2023-11-24T06:24:19.866835+0000","last_active":"2023-11-24T06:24:20.854899+0000","last_peered":"2023-11-24T06:24:20.854899+0000","last_clean":"2023-11-24T06:24:20.854899+0000","last_became_active":"2023-11-24T06:24:19.866476+0000","last_became_peered":"2023-11-24T06:24:19.866476+0000","last_unstale":"2023-11-24T06:24:20.854899+0000","last_undegraded":"2023-11-24T06:24:20.854899+0000","last_fullsized":"2023-11-24T06:24:20.854899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:20.926427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553269+0000","last_change":"2023-11-24T06:24:19.873151+0000","last_active":"2023-11-24T06:24:21.553269+0000","last_peered":"2023-11-24T06:24:21.553269+0000","last_clean":"2023-11-24T06:24:21.553269+0000","last_became_active":"2023-11-24T06:24:19.872848+0000","last_became_peered":"2023-11-24T06:24:19.872848+0000","last_unstale":"2023-11-24T06:24:21.553269+0000","last_undegraded":"2023-11-24T06:24:21.553269+0000","last_fullsized":"2023-11-24T06:24:21.553269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553077+0000","last_change":"2023-11-24T06:24:19.866025+0000","last_active":"2023-11-24T06:24:21.553077+0000","last_peered":"2023-11-24T06:24:21.553077+0000","last_clean":"2023-11-24T06:24:21.553077+0000","last_became_active":"2023-11-24T06:24:19.865501+0000","last_became_peered":"2023-11-24T06:24:19.865501+0000","last_unstale":"2023-11-24T06:24:21.553077+0000","last_undegraded":"2023-11-24T06:24:21.553077+0000","last_fullsized":"2023-11-24T06:24:21.553077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054789+0000","last_change":"2023-11-24T06:24:19.876131+0000","last_active":"2023-11-24T06:24:21.054789+0000","last_peered":"2023-11-24T06:24:21.054789+0000","last_clean":"2023-11-24T06:24:21.054789+0000","last_became_active":"2023-11-24T06:24:19.875876+0000","last_became_peered":"2023-11-24T06:24:19.875876+0000","last_unstale":"2023-11-24T06:24:21.054789+0000","last_undegraded":"2023-11-24T06:24:21.054789+0000","last_fullsized":"2023-11-24T06:24:21.054789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979358+0000","last_change":"2023-11-24T06:24:19.865439+0000","last_active":"2023-11-24T06:24:20.979358+0000","last_peered":"2023-11-24T06:24:20.979358+0000","last_clean":"2023-11-24T06:24:20.979358+0000","last_became_active":"2023-11-24T06:24:19.865124+0000","last_became_peered":"2023-11-24T06:24:19.865124+0000","last_unstale":"2023-11-24T06:24:20.979358+0000","last_undegraded":"2023-11-24T06:24:20.979358+0000","last_fullsized":"2023-11-24T06:24:20.979358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904352+0000","last_change":"2023-11-24T06:24:19.865010+0000","last_active":"2023-11-24T06:24:20.904352+0000","last_peered":"2023-11-24T06:24:20.904352+0000","last_clean":"2023-11-24T06:24:20.904352+0000","last_became_active":"2023-11-24T06:24:19.864730+0000","last_became_peered":"2023-11-24T06:24:19.864730+0000","last_unstale":"2023-11-24T06:24:20.904352+0000","last_undegraded":"2023-11-24T06:24:20.904352+0000","last_fullsized":"2023-11-24T06:24:20.904352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853108+0000","last_change":"2023-11-24T06:24:19.873933+0000","last_active":"2023-11-24T06:24:20.853108+0000","last_peered":"2023-11-24T06:24:20.853108+0000","last_clean":"2023-11-24T06:24:20.853108+0000","last_became_active":"2023-11-24T06:24:19.873679+0000","last_became_peered":"2023-11-24T06:24:19.873679+0000","last_unstale":"2023-11-24T06:24:20.853108+0000","last_undegraded":"2023-11-24T06:24:20.853108+0000","last_fullsized":"2023-11-24T06:24:20.853108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553458+0000","last_change":"2023-11-24T06:24:19.873987+0000","last_active":"2023-11-24T06:24:21.553458+0000","last_peered":"2023-11-24T06:24:21.553458+0000","last_clean":"2023-11-24T06:24:21.553458+0000","last_became_active":"2023-11-24T06:24:19.871835+0000","last_became_peered":"2023-11-24T06:24:19.871835+0000","last_unstale":"2023-11-24T06:24:21.553458+0000","last_undegraded":"2023-11-24T06:24:21.553458+0000","last_fullsized":"2023-11-24T06:24:21.553458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904255+0000","last_change":"2023-11-24T06:24:19.863300+0000","last_active":"2023-11-24T06:24:20.904255+0000","last_peered":"2023-11-24T06:24:20.904255+0000","last_clean":"2023-11-24T06:24:20.904255+0000","last_became_active":"2023-11-24T06:24:19.861918+0000","last_became_peered":"2023-11-24T06:24:19.861918+0000","last_unstale":"2023-11-24T06:24:20.904255+0000","last_undegraded":"2023-11-24T06:24:20.904255+0000","last_fullsized":"2023-11-24T06:24:20.904255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054464+0000","last_change":"2023-11-24T06:24:19.878153+0000","last_active":"2023-11-24T06:24:21.054464+0000","last_peered":"2023-11-24T06:24:21.054464+0000","last_clean":"2023-11-24T06:24:21.054464+0000","last_became_active":"2023-11-24T06:24:19.876482+0000","last_became_peered":"2023-11-24T06:24:19.876482+0000","last_unstale":"2023-11-24T06:24:21.054464+0000","last_undegraded":"2023-11-24T06:24:21.054464+0000","last_fullsized":"2023-11-24T06:24:21.054464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904064+0000","last_change":"2023-11-24T06:24:19.862468+0000","last_active":"2023-11-24T06:24:20.904064+0000","last_peered":"2023-11-24T06:24:20.904064+0000","last_clean":"2023-11-24T06:24:20.904064+0000","last_became_active":"2023-11-24T06:24:19.862178+0000","last_became_peered":"2023-11-24T06:24:19.862178+0000","last_unstale":"2023-11-24T06:24:20.904064+0000","last_undegraded":"2023-11-24T06:24:20.904064+0000","last_fullsized":"2023-11-24T06:24:20.904064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054544+0000","last_change":"2023-11-24T06:24:19.871577+0000","last_active":"2023-11-24T06:24:21.054544+0000","last_peered":"2023-11-24T06:24:21.054544+0000","last_clean":"2023-11-24T06:24:21.054544+0000","last_became_active":"2023-11-24T06:24:19.871279+0000","last_became_peered":"2023-11-24T06:24:19.871279+0000","last_unstale":"2023-11-24T06:24:21.054544+0000","last_undegraded":"2023-11-24T06:24:21.054544+0000","last_fullsized":"2023-11-24T06:24:21.054544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979353+0000","last_change":"2023-11-24T06:24:19.872608+0000","last_active":"2023-11-24T06:24:20.979353+0000","last_peered":"2023-11-24T06:24:20.979353+0000","last_clean":"2023-11-24T06:24:20.979353+0000","last_became_active":"2023-11-24T06:24:19.872343+0000","last_became_peered":"2023-11-24T06:24:19.872343+0000","last_unstale":"2023-11-24T06:24:20.979353+0000","last_undegraded":"2023-11-24T06:24:20.979353+0000","last_fullsized":"2023-11-24T06:24:20.979353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553301+0000","last_change":"2023-11-24T06:24:19.868776+0000","last_active":"2023-11-24T06:24:21.553301+0000","last_peered":"2023-11-24T06:24:21.553301+0000","last_clean":"2023-11-24T06:24:21.553301+0000","last_became_active":"2023-11-24T06:24:19.868335+0000","last_became_peered":"2023-11-24T06:24:19.868335+0000","last_unstale":"2023-11-24T06:24:21.553301+0000","last_undegraded":"2023-11-24T06:24:21.553301+0000","last_fullsized":"2023-11-24T06:24:21.553301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:33.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:24:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.331+0000 7f86fa1ef700 1 Processor -- start 2023-11-24T06:24:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.331+0000 7f86fa1ef700 1 -- start start 2023-11-24T06:24:33.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86fa1ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f40b3370 0x7f86f41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86fa1ef700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86f40d1c70 0x7f86f40d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86fa1ef700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f40b22d0 con 0x7f86f415bdd0 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86fa1ef700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f40b2f50 con 0x7f86f40d1c70 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86fa1ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40d2640 con 0x7f86f40b3370 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86f40d1c70 0x7f86f40d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86f40d1c70 0x7f86f40d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38298/0 (socket says 172.21.15.114:38298) 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 -- 172.21.15.114:0/2344296769 learned_addr learned my addr 172.21.15.114:0/2344296769 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f2ffd700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f40b3370 0x7f86f41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 -- 172.21.15.114:0/2344296769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f40b3370 msgr2=0x7f86f41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f40b3370 0x7f86f41604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 -- 172.21.15.114:0/2344296769 >> v1:172.21.15.172:6789/0 conn(0x7f86f415bdd0 legacy=0x7f86f40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 -- 172.21.15.114:0/2344296769 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40b25b0 con 0x7f86f40d1c70 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.332+0000 7f86f3fff700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40d1c70 0x7f86f40d2100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009e50 tx=0x7f86ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3bc6784bdbd16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86f27fc700 1 -- 172.21.15.114:0/2344296769 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00e070 con 0x7f86f40d1c70 2023-11-24T06:24:33.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86f27fc700 1 -- 172.21.15.114:0/2344296769 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec003a20 con 0x7f86f40d1c70 2023-11-24T06:24:33.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86f27fc700 1 -- 172.21.15.114:0/2344296769 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec003dd0 con 0x7f86f40d1c70 2023-11-24T06:24:33.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2344296769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40d1c70 msgr2=0x7f86f40d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40d1c70 0x7f86f40d2100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009e50 tx=0x7f86ec00b330 comp rx=0 tx=0).stop 2023-11-24T06:24:33.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2344296769 shutdown_connections 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40d1c70 0x7f86f40d2100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2344296769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f40b3370 0x7f86f41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2344296769 >> 172.21.15.114:0/2344296769 conn(0x7f86f40731a0 msgr2=0x7f86f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2344296769 shutdown_connections 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.333+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2344296769 wait complete. 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.334+0000 7f86fa1ef700 1 Processor -- start 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 -- start start 2023-11-24T06:24:33.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f40d1c70 0x7f86f40b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f415bdd0 0x7f86f40b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f415e9b0 con 0x7f86f40b3370 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b5c50 con 0x7f86f40d1c70 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86fa1ef700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b5f30 con 0x7f86f415bdd0 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38302/0 (socket says 172.21.15.114:38302) 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86f37fe700 1 -- 172.21.15.114:0/2420086413 learned_addr learned my addr 172.21.15.114:0/2420086413 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86f3fff700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f415bdd0 0x7f86f40b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.335+0000 7f86f2ffd700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f40d1c70 0x7f86f40b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 -- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f415bdd0 msgr2=0x7f86f40b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f415bdd0 0x7f86f40b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 -- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f40d1c70 msgr2=0x7f86f40b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f40d1c70 0x7f86f40b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 -- 172.21.15.114:0/2420086413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec009b30 con 0x7f86f40b3370 2023-11-24T06:24:33.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86f37fe700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b4a0 tx=0x7f86e800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8011830 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8011990 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f40b6210 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8011d40 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.336+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f40b67d0 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.337+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f4067f60 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.337+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86e801a400 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.338+0000 7f86dbfff700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 0x7f86e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.338+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86e8089220 con 0x7f86f40b3370 2023-11-24T06:24:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.338+0000 7f86f2ffd700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 0x7f86e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.338+0000 7f86f2ffd700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 0x7f86e006d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f86f40703f0 tx=0x7f86e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.342+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86e8054410 con 0x7f86f40b3370 2023-11-24T06:24:33.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.528+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86f40b2b70 con 0x7f86e006aed0 2023-11-24T06:24:33.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.535+0000 7f86dbfff700 1 -- 172.21.15.114:0/2420086413 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264607 (secure 0 0 0) 0x7f86e400a680 con 0x7f86e006aed0 2023-11-24T06:24:33.541 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:33.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 msgr2=0x7f86e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 0x7f86e006d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f86f40703f0 tx=0x7f86e4008040 comp rx=0 tx=0).stop 2023-11-24T06:24:33.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 msgr2=0x7f86f40b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b4a0 tx=0x7f86e800b860 comp rx=0 tx=0).stop 2023-11-24T06:24:33.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 shutdown_connections 2023-11-24T06:24:33.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86e006aed0 0x7f86e006d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86f415bdd0 0x7f86f40b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86f40b3370 0x7f86f40b6cf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 --2- 172.21.15.114:0/2420086413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86f40d1c70 0x7f86f40b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 >> 172.21.15.114:0/2420086413 conn(0x7f86f40731a0 msgr2=0x7f86f40b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:33.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 shutdown_connections 2023-11-24T06:24:33.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.538+0000 7f86fa1ef700 1 -- 172.21.15.114:0/2420086413 wait complete. 2023-11-24T06:24:33.550 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:24:33.554 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-24T06:24:31.710230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219108,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755612,"statfs":{"total":773094113280,"available":772869746688,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.858893"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854842+0000","last_change":"2023-11-24T06:24:19.869889+0000","last_active":"2023-11-24T06:24:20.854842+0000","last_peered":"2023-11-24T06:24:20.854842+0000","last_clean":"2023-11-24T06:24:20.854842+0000","last_became_active":"2023-11-24T06:24:19.869608+0000","last_became_peered":"2023-11-24T06:24:19.869608+0000","last_unstale":"2023-11-24T06:24:20.854842+0000","last_undegraded":"2023-11-24T06:24:20.854842+0000","last_fullsized":"2023-11-24T06:24:20.854842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:48.551881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853246+0000","last_change":"2023-11-24T06:24:19.872814+0000","last_active":"2023-11-24T06:24:20.853246+0000","last_peered":"2023-11-24T06:24:20.853246+0000","last_clean":"2023-11-24T06:24:20.853246+0000","last_became_active":"2023-11-24T06:24:19.872430+0000","last_became_peered":"2023-11-24T06:24:19.872430+0000","last_unstale":"2023-11-24T06:24:20.853246+0000","last_undegraded":"2023-11-24T06:24:20.853246+0000","last_fullsized":"2023-11-24T06:24:20.853246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979460+0000","last_change":"2023-11-24T06:24:19.869057+0000","last_active":"2023-11-24T06:24:20.979460+0000","last_peered":"2023-11-24T06:24:20.979460+0000","last_clean":"2023-11-24T06:24:20.979460+0000","last_became_active":"2023-11-24T06:24:19.868779+0000","last_became_peered":"2023-11-24T06:24:19.868779+0000","last_unstale":"2023-11-24T06:24:20.979460+0000","last_undegraded":"2023-11-24T06:24:20.979460+0000","last_fullsized":"2023-11-24T06:24:20.979460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054845+0000","last_change":"2023-11-24T06:24:19.875414+0000","last_active":"2023-11-24T06:24:21.054845+0000","last_peered":"2023-11-24T06:24:21.054845+0000","last_clean":"2023-11-24T06:24:21.054845+0000","last_became_active":"2023-11-24T06:24:19.875156+0000","last_became_peered":"2023-11-24T06:24:19.875156+0000","last_unstale":"2023-11-24T06:24:21.054845+0000","last_undegraded":"2023-11-24T06:24:21.054845+0000","last_fullsized":"2023-11-24T06:24:21.054845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854274+0000","last_change":"2023-11-24T06:24:19.862612+0000","last_active":"2023-11-24T06:24:20.854274+0000","last_peered":"2023-11-24T06:24:20.854274+0000","last_clean":"2023-11-24T06:24:20.854274+0000","last_became_active":"2023-11-24T06:24:19.862302+0000","last_became_peered":"2023-11-24T06:24:19.862302+0000","last_unstale":"2023-11-24T06:24:20.854274+0000","last_undegraded":"2023-11-24T06:24:20.854274+0000","last_fullsized":"2023-11-24T06:24:20.854274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054394+0000","last_change":"2023-11-24T06:24:19.871642+0000","last_active":"2023-11-24T06:24:21.054394+0000","last_peered":"2023-11-24T06:24:21.054394+0000","last_clean":"2023-11-24T06:24:21.054394+0000","last_became_active":"2023-11-24T06:24:19.871363+0000","last_became_peered":"2023-11-24T06:24:19.871363+0000","last_unstale":"2023-11-24T06:24:21.054394+0000","last_undegraded":"2023-11-24T06:24:21.054394+0000","last_fullsized":"2023-11-24T06:24:21.054394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854102+0000","last_change":"2023-11-24T06:24:19.865691+0000","last_active":"2023-11-24T06:24:20.854102+0000","last_peered":"2023-11-24T06:24:20.854102+0000","last_clean":"2023-11-24T06:24:20.854102+0000","last_became_active":"2023-11-24T06:24:19.865409+0000","last_became_peered":"2023-11-24T06:24:19.865409+0000","last_unstale":"2023-11-24T06:24:20.854102+0000","last_undegraded":"2023-11-24T06:24:20.854102+0000","last_fullsized":"2023-11-24T06:24:20.854102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904022+0000","last_change":"2023-11-24T06:24:19.866800+0000","last_active":"2023-11-24T06:24:20.904022+0000","last_peered":"2023-11-24T06:24:20.904022+0000","last_clean":"2023-11-24T06:24:20.904022+0000","last_became_active":"2023-11-24T06:24:19.866162+0000","last_became_peered":"2023-11-24T06:24:19.866162+0000","last_unstale":"2023-11-24T06:24:20.904022+0000","last_undegraded":"2023-11-24T06:24:20.904022+0000","last_fullsized":"2023-11-24T06:24:20.904022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854764+0000","last_change":"2023-11-24T06:24:19.867736+0000","last_active":"2023-11-24T06:24:20.854764+0000","last_peered":"2023-11-24T06:24:20.854764+0000","last_clean":"2023-11-24T06:24:20.854764+0000","last_became_active":"2023-11-24T06:24:19.867377+0000","last_became_peered":"2023-11-24T06:24:19.867377+0000","last_unstale":"2023-11-24T06:24:20.854764+0000","last_undegraded":"2023-11-24T06:24:20.854764+0000","last_fullsized":"2023-11-24T06:24:20.854764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553520+0000","last_change":"2023-11-24T06:24:19.870504+0000","last_active":"2023-11-24T06:24:21.553520+0000","last_peered":"2023-11-24T06:24:21.553520+0000","last_clean":"2023-11-24T06:24:21.553520+0000","last_became_active":"2023-11-24T06:24:19.870143+0000","last_became_peered":"2023-11-24T06:24:19.870143+0000","last_unstale":"2023-11-24T06:24:21.553520+0000","last_undegraded":"2023-11-24T06:24:21.553520+0000","last_fullsized":"2023-11-24T06:24:21.553520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553151+0000","last_change":"2023-11-24T06:24:19.873638+0000","last_active":"2023-11-24T06:24:21.553151+0000","last_peered":"2023-11-24T06:24:21.553151+0000","last_clean":"2023-11-24T06:24:21.553151+0000","last_became_active":"2023-11-24T06:24:19.871551+0000","last_became_peered":"2023-11-24T06:24:19.871551+0000","last_unstale":"2023-11-24T06:24:21.553151+0000","last_undegraded":"2023-11-24T06:24:21.553151+0000","last_fullsized":"2023-11-24T06:24:21.553151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855889+0000","last_change":"2023-11-24T06:24:19.867808+0000","last_active":"2023-11-24T06:24:20.855889+0000","last_peered":"2023-11-24T06:24:20.855889+0000","last_clean":"2023-11-24T06:24:20.855889+0000","last_became_active":"2023-11-24T06:24:19.867487+0000","last_became_peered":"2023-11-24T06:24:19.867487+0000","last_unstale":"2023-11-24T06:24:20.855889+0000","last_undegraded":"2023-11-24T06:24:20.855889+0000","last_fullsized":"2023-11-24T06:24:20.855889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904066+0000","last_change":"2023-11-24T06:24:19.870784+0000","last_active":"2023-11-24T06:24:20.904066+0000","last_peered":"2023-11-24T06:24:20.904066+0000","last_clean":"2023-11-24T06:24:20.904066+0000","last_became_active":"2023-11-24T06:24:19.870374+0000","last_became_peered":"2023-11-24T06:24:19.870374+0000","last_unstale":"2023-11-24T06:24:20.904066+0000","last_undegraded":"2023-11-24T06:24:20.904066+0000","last_fullsized":"2023-11-24T06:24:20.904066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979266+0000","last_change":"2023-11-24T06:24:19.865517+0000","last_active":"2023-11-24T06:24:20.979266+0000","last_peered":"2023-11-24T06:24:20.979266+0000","last_clean":"2023-11-24T06:24:20.979266+0000","last_became_active":"2023-11-24T06:24:19.865195+0000","last_became_peered":"2023-11-24T06:24:19.865195+0000","last_unstale":"2023-11-24T06:24:20.979266+0000","last_undegraded":"2023-11-24T06:24:20.979266+0000","last_fullsized":"2023-11-24T06:24:20.979266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852902+0000","last_change":"2023-11-24T06:24:19.867596+0000","last_active":"2023-11-24T06:24:20.852902+0000","last_peered":"2023-11-24T06:24:20.852902+0000","last_clean":"2023-11-24T06:24:20.852902+0000","last_became_active":"2023-11-24T06:24:19.867302+0000","last_became_peered":"2023-11-24T06:24:19.867302+0000","last_unstale":"2023-11-24T06:24:20.852902+0000","last_undegraded":"2023-11-24T06:24:20.852902+0000","last_fullsized":"2023-11-24T06:24:20.852902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:49.739892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054302+0000","last_change":"2023-11-24T06:24:19.871929+0000","last_active":"2023-11-24T06:24:21.054302+0000","last_peered":"2023-11-24T06:24:21.054302+0000","last_clean":"2023-11-24T06:24:21.054302+0000","last_became_active":"2023-11-24T06:24:19.870555+0000","last_became_peered":"2023-11-24T06:24:19.870555+0000","last_unstale":"2023-11-24T06:24:21.054302+0000","last_undegraded":"2023-11-24T06:24:21.054302+0000","last_fullsized":"2023-11-24T06:24:21.054302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054970+0000","last_change":"2023-11-24T06:24:19.871912+0000","last_active":"2023-11-24T06:24:21.054970+0000","last_peered":"2023-11-24T06:24:21.054970+0000","last_clean":"2023-11-24T06:24:21.054970+0000","last_became_active":"2023-11-24T06:24:19.871473+0000","last_became_peered":"2023-11-24T06:24:19.871473+0000","last_unstale":"2023-11-24T06:24:21.054970+0000","last_undegraded":"2023-11-24T06:24:21.054970+0000","last_fullsized":"2023-11-24T06:24:21.054970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:43.758754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553079+0000","last_change":"2023-11-24T06:24:19.873774+0000","last_active":"2023-11-24T06:24:21.553079+0000","last_peered":"2023-11-24T06:24:21.553079+0000","last_clean":"2023-11-24T06:24:21.553079+0000","last_became_active":"2023-11-24T06:24:19.872880+0000","last_became_peered":"2023-11-24T06:24:19.872880+0000","last_unstale":"2023-11-24T06:24:21.553079+0000","last_undegraded":"2023-11-24T06:24:21.553079+0000","last_fullsized":"2023-11-24T06:24:21.553079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854750+0000","last_change":"2023-11-24T06:24:19.863508+0000","last_active":"2023-11-24T06:24:20.854750+0000","last_peered":"2023-11-24T06:24:20.854750+0000","last_clean":"2023-11-24T06:24:20.854750+0000","last_became_active":"2023-11-24T06:24:19.863100+0000","last_became_peered":"2023-11-24T06:24:19.863100+0000","last_unstale":"2023-11-24T06:24:20.854750+0000","last_undegraded":"2023-11-24T06:24:20.854750+0000","last_fullsized":"2023-11-24T06:24:20.854750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:19.815953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.892967+0000","last_change":"2023-11-24T06:24:18.857987+0000","last_active":"2023-11-24T06:24:19.892967+0000","last_peered":"2023-11-24T06:24:19.892967+0000","last_clean":"2023-11-24T06:24:19.892967+0000","last_became_active":"2023-11-24T06:24:18.857731+0000","last_became_peered":"2023-11-24T06:24:18.857731+0000","last_unstale":"2023-11-24T06:24:19.892967+0000","last_undegraded":"2023-11-24T06:24:19.892967+0000","last_fullsized":"2023-11-24T06:24:19.892967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904290+0000","last_change":"2023-11-24T06:24:19.869582+0000","last_active":"2023-11-24T06:24:20.904290+0000","last_peered":"2023-11-24T06:24:20.904290+0000","last_clean":"2023-11-24T06:24:20.904290+0000","last_became_active":"2023-11-24T06:24:19.869283+0000","last_became_peered":"2023-11-24T06:24:19.869283+0000","last_unstale":"2023-11-24T06:24:20.904290+0000","last_undegraded":"2023-11-24T06:24:20.904290+0000","last_fullsized":"2023-11-24T06:24:20.904290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883379+0000","last_change":"2023-11-24T06:24:18.857370+0000","last_active":"2023-11-24T06:24:19.883379+0000","last_peered":"2023-11-24T06:24:19.883379+0000","last_clean":"2023-11-24T06:24:19.883379+0000","last_became_active":"2023-11-24T06:24:18.857081+0000","last_became_peered":"2023-11-24T06:24:18.857081+0000","last_unstale":"2023-11-24T06:24:19.883379+0000","last_undegraded":"2023-11-24T06:24:19.883379+0000","last_fullsized":"2023-11-24T06:24:19.883379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979537+0000","last_change":"2023-11-24T06:24:19.869681+0000","last_active":"2023-11-24T06:24:20.979537+0000","last_peered":"2023-11-24T06:24:20.979537+0000","last_clean":"2023-11-24T06:24:20.979537+0000","last_became_active":"2023-11-24T06:24:19.869437+0000","last_became_peered":"2023-11-24T06:24:19.869437+0000","last_unstale":"2023-11-24T06:24:20.979537+0000","last_undegraded":"2023-11-24T06:24:20.979537+0000","last_fullsized":"2023-11-24T06:24:20.979537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879979+0000","last_change":"2023-11-24T06:24:18.860999+0000","last_active":"2023-11-24T06:24:19.879979+0000","last_peered":"2023-11-24T06:24:19.879979+0000","last_clean":"2023-11-24T06:24:19.879979+0000","last_became_active":"2023-11-24T06:24:18.860405+0000","last_became_peered":"2023-11-24T06:24:18.860405+0000","last_unstale":"2023-11-24T06:24:19.879979+0000","last_undegraded":"2023-11-24T06:24:19.879979+0000","last_fullsized":"2023-11-24T06:24:19.879979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:29.087819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054525+0000","last_change":"2023-11-24T06:24:19.869851+0000","last_active":"2023-11-24T06:24:21.054525+0000","last_peered":"2023-11-24T06:24:21.054525+0000","last_clean":"2023-11-24T06:24:21.054525+0000","last_became_active":"2023-11-24T06:24:19.869550+0000","last_became_peered":"2023-11-24T06:24:19.869550+0000","last_unstale":"2023-11-24T06:24:21.054525+0000","last_undegraded":"2023-11-24T06:24:21.054525+0000","last_fullsized":"2023-11-24T06:24:21.054525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:35:29.880535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880299+0000","last_change":"2023-11-24T06:24:18.862616+0000","last_active":"2023-11-24T06:24:19.880299+0000","last_peered":"2023-11-24T06:24:19.880299+0000","last_clean":"2023-11-24T06:24:19.880299+0000","last_became_active":"2023-11-24T06:24:18.862321+0000","last_became_peered":"2023-11-24T06:24:18.862321+0000","last_unstale":"2023-11-24T06:24:19.880299+0000","last_undegraded":"2023-11-24T06:24:19.880299+0000","last_fullsized":"2023-11-24T06:24:19.880299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054853+0000","last_change":"2023-11-24T06:24:19.870442+0000","last_active":"2023-11-24T06:24:21.054853+0000","last_peered":"2023-11-24T06:24:21.054853+0000","last_clean":"2023-11-24T06:24:21.054853+0000","last_became_active":"2023-11-24T06:24:19.870164+0000","last_became_peered":"2023-11-24T06:24:19.870164+0000","last_unstale":"2023-11-24T06:24:21.054853+0000","last_undegraded":"2023-11-24T06:24:21.054853+0000","last_fullsized":"2023-11-24T06:24:21.054853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883377+0000","last_change":"2023-11-24T06:24:18.861487+0000","last_active":"2023-11-24T06:24:19.883377+0000","last_peered":"2023-11-24T06:24:19.883377+0000","last_clean":"2023-11-24T06:24:19.883377+0000","last_became_active":"2023-11-24T06:24:18.861225+0000","last_became_peered":"2023-11-24T06:24:18.861225+0000","last_unstale":"2023-11-24T06:24:19.883377+0000","last_undegraded":"2023-11-24T06:24:19.883377+0000","last_fullsized":"2023-11-24T06:24:19.883377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:18.941579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855381+0000","last_change":"2023-11-24T06:24:19.864863+0000","last_active":"2023-11-24T06:24:20.855381+0000","last_peered":"2023-11-24T06:24:20.855381+0000","last_clean":"2023-11-24T06:24:20.855381+0000","last_became_active":"2023-11-24T06:24:19.864621+0000","last_became_peered":"2023-11-24T06:24:19.864621+0000","last_unstale":"2023-11-24T06:24:20.855381+0000","last_undegraded":"2023-11-24T06:24:20.855381+0000","last_fullsized":"2023-11-24T06:24:20.855381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:51.049403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879363+0000","last_change":"2023-11-24T06:24:18.854038+0000","last_active":"2023-11-24T06:24:19.879363+0000","last_peered":"2023-11-24T06:24:19.879363+0000","last_clean":"2023-11-24T06:24:19.879363+0000","last_became_active":"2023-11-24T06:24:18.853760+0000","last_became_peered":"2023-11-24T06:24:18.853760+0000","last_unstale":"2023-11-24T06:24:19.879363+0000","last_undegraded":"2023-11-24T06:24:19.879363+0000","last_fullsized":"2023-11-24T06:24:19.879363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979510+0000","last_change":"2023-11-24T06:24:19.866029+0000","last_active":"2023-11-24T06:24:20.979510+0000","last_peered":"2023-11-24T06:24:20.979510+0000","last_clean":"2023-11-24T06:24:20.979510+0000","last_became_active":"2023-11-24T06:24:19.865361+0000","last_became_peered":"2023-11-24T06:24:19.865361+0000","last_unstale":"2023-11-24T06:24:20.979510+0000","last_undegraded":"2023-11-24T06:24:20.979510+0000","last_fullsized":"2023-11-24T06:24:20.979510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852200+0000","last_change":"2023-11-24T06:24:18.857296+0000","last_active":"2023-11-24T06:24:19.852200+0000","last_peered":"2023-11-24T06:24:19.852200+0000","last_clean":"2023-11-24T06:24:19.852200+0000","last_became_active":"2023-11-24T06:24:18.857021+0000","last_became_peered":"2023-11-24T06:24:18.857021+0000","last_unstale":"2023-11-24T06:24:19.852200+0000","last_undegraded":"2023-11-24T06:24:19.852200+0000","last_fullsized":"2023-11-24T06:24:19.852200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:19.237345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904187+0000","last_change":"2023-11-24T06:24:19.863134+0000","last_active":"2023-11-24T06:24:20.904187+0000","last_peered":"2023-11-24T06:24:20.904187+0000","last_clean":"2023-11-24T06:24:20.904187+0000","last_became_active":"2023-11-24T06:24:19.862679+0000","last_became_peered":"2023-11-24T06:24:19.862679+0000","last_unstale":"2023-11-24T06:24:20.904187+0000","last_undegraded":"2023-11-24T06:24:20.904187+0000","last_fullsized":"2023-11-24T06:24:20.904187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.862408+0000","last_change":"2023-11-24T06:24:18.858707+0000","last_active":"2023-11-24T06:24:19.862408+0000","last_peered":"2023-11-24T06:24:19.862408+0000","last_clean":"2023-11-24T06:24:19.862408+0000","last_became_active":"2023-11-24T06:24:18.858447+0000","last_became_peered":"2023-11-24T06:24:18.858447+0000","last_unstale":"2023-11-24T06:24:19.862408+0000","last_undegraded":"2023-11-24T06:24:19.862408+0000","last_fullsized":"2023-11-24T06:24:19.862408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:38.777737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853050+0000","last_change":"2023-11-24T06:24:19.864439+0000","last_active":"2023-11-24T06:24:20.853050+0000","last_peered":"2023-11-24T06:24:20.853050+0000","last_clean":"2023-11-24T06:24:20.853050+0000","last_became_active":"2023-11-24T06:24:19.864128+0000","last_became_peered":"2023-11-24T06:24:19.864128+0000","last_unstale":"2023-11-24T06:24:20.853050+0000","last_undegraded":"2023-11-24T06:24:20.853050+0000","last_fullsized":"2023-11-24T06:24:20.853050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860879+0000","last_change":"2023-11-24T06:24:18.862376+0000","last_active":"2023-11-24T06:24:19.860879+0000","last_peered":"2023-11-24T06:24:19.860879+0000","last_clean":"2023-11-24T06:24:19.860879+0000","last_became_active":"2023-11-24T06:24:18.862070+0000","last_became_peered":"2023-11-24T06:24:18.862070+0000","last_unstale":"2023-11-24T06:24:19.860879+0000","last_undegraded":"2023-11-24T06:24:19.860879+0000","last_fullsized":"2023-11-24T06:24:19.860879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054843+0000","last_change":"2023-11-24T06:24:19.874520+0000","last_active":"2023-11-24T06:24:21.054843+0000","last_peered":"2023-11-24T06:24:21.054843+0000","last_clean":"2023-11-24T06:24:21.054843+0000","last_became_active":"2023-11-24T06:24:19.872481+0000","last_became_peered":"2023-11-24T06:24:19.872481+0000","last_unstale":"2023-11-24T06:24:21.054843+0000","last_undegraded":"2023-11-24T06:24:21.054843+0000","last_fullsized":"2023-11-24T06:24:21.054843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.865095+0000","last_change":"2023-11-24T06:24:18.858132+0000","last_active":"2023-11-24T06:24:19.865095+0000","last_peered":"2023-11-24T06:24:19.865095+0000","last_clean":"2023-11-24T06:24:19.865095+0000","last_became_active":"2023-11-24T06:24:18.857854+0000","last_became_peered":"2023-11-24T06:24:18.857854+0000","last_unstale":"2023-11-24T06:24:19.865095+0000","last_undegraded":"2023-11-24T06:24:19.865095+0000","last_fullsized":"2023-11-24T06:24:19.865095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:51.886938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979608+0000","last_change":"2023-11-24T06:24:19.867545+0000","last_active":"2023-11-24T06:24:20.979608+0000","last_peered":"2023-11-24T06:24:20.979608+0000","last_clean":"2023-11-24T06:24:20.979608+0000","last_became_active":"2023-11-24T06:24:19.867313+0000","last_became_peered":"2023-11-24T06:24:19.867313+0000","last_unstale":"2023-11-24T06:24:20.979608+0000","last_undegraded":"2023-11-24T06:24:20.979608+0000","last_fullsized":"2023-11-24T06:24:20.979608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885062+0000","last_change":"2023-11-24T06:24:18.864992+0000","last_active":"2023-11-24T06:24:19.885062+0000","last_peered":"2023-11-24T06:24:19.885062+0000","last_clean":"2023-11-24T06:24:19.885062+0000","last_became_active":"2023-11-24T06:24:18.864701+0000","last_became_peered":"2023-11-24T06:24:18.864701+0000","last_unstale":"2023-11-24T06:24:19.885062+0000","last_undegraded":"2023-11-24T06:24:19.885062+0000","last_fullsized":"2023-11-24T06:24:19.885062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:08.350203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855876+0000","last_change":"2023-11-24T06:24:19.864361+0000","last_active":"2023-11-24T06:24:20.855876+0000","last_peered":"2023-11-24T06:24:20.855876+0000","last_clean":"2023-11-24T06:24:20.855876+0000","last_became_active":"2023-11-24T06:24:19.864077+0000","last_became_peered":"2023-11-24T06:24:19.864077+0000","last_unstale":"2023-11-24T06:24:20.855876+0000","last_undegraded":"2023-11-24T06:24:20.855876+0000","last_fullsized":"2023-11-24T06:24:20.855876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:05.886558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885718+0000","last_change":"2023-11-24T06:24:18.860308+0000","last_active":"2023-11-24T06:24:19.885718+0000","last_peered":"2023-11-24T06:24:19.885718+0000","last_clean":"2023-11-24T06:24:19.885718+0000","last_became_active":"2023-11-24T06:24:18.860009+0000","last_became_peered":"2023-11-24T06:24:18.860009+0000","last_unstale":"2023-11-24T06:24:19.885718+0000","last_undegraded":"2023-11-24T06:24:19.885718+0000","last_fullsized":"2023-11-24T06:24:19.885718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854301+0000","last_change":"2023-11-24T06:24:19.858071+0000","last_active":"2023-11-24T06:24:20.854301+0000","last_peered":"2023-11-24T06:24:20.854301+0000","last_clean":"2023-11-24T06:24:20.854301+0000","last_became_active":"2023-11-24T06:24:19.857803+0000","last_became_peered":"2023-11-24T06:24:19.857803+0000","last_unstale":"2023-11-24T06:24:20.854301+0000","last_undegraded":"2023-11-24T06:24:20.854301+0000","last_fullsized":"2023-11-24T06:24:20.854301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.882906+0000","last_change":"2023-11-24T06:24:18.851973+0000","last_active":"2023-11-24T06:24:19.882906+0000","last_peered":"2023-11-24T06:24:19.882906+0000","last_clean":"2023-11-24T06:24:19.882906+0000","last_became_active":"2023-11-24T06:24:18.851683+0000","last_became_peered":"2023-11-24T06:24:18.851683+0000","last_unstale":"2023-11-24T06:24:19.882906+0000","last_undegraded":"2023-11-24T06:24:19.882906+0000","last_fullsized":"2023-11-24T06:24:19.882906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:05.954261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979188+0000","last_change":"2023-11-24T06:24:19.870356+0000","last_active":"2023-11-24T06:24:20.979188+0000","last_peered":"2023-11-24T06:24:20.979188+0000","last_clean":"2023-11-24T06:24:20.979188+0000","last_became_active":"2023-11-24T06:24:19.870057+0000","last_became_peered":"2023-11-24T06:24:19.870057+0000","last_unstale":"2023-11-24T06:24:20.979188+0000","last_undegraded":"2023-11-24T06:24:20.979188+0000","last_fullsized":"2023-11-24T06:24:20.979188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:30.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864248+0000","last_change":"2023-11-24T06:24:18.859626+0000","last_active":"2023-11-24T06:24:19.864248+0000","last_peered":"2023-11-24T06:24:19.864248+0000","last_clean":"2023-11-24T06:24:19.864248+0000","last_became_active":"2023-11-24T06:24:18.859305+0000","last_became_peered":"2023-11-24T06:24:18.859305+0000","last_unstale":"2023-11-24T06:24:19.864248+0000","last_undegraded":"2023-11-24T06:24:19.864248+0000","last_fullsized":"2023-11-24T06:24:19.864248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854429+0000","last_change":"2023-11-24T06:24:19.863989+0000","last_active":"2023-11-24T06:24:20.854429+0000","last_peered":"2023-11-24T06:24:20.854429+0000","last_clean":"2023-11-24T06:24:20.854429+0000","last_became_active":"2023-11-24T06:24:19.861502+0000","last_became_peered":"2023-11-24T06:24:19.861502+0000","last_unstale":"2023-11-24T06:24:20.854429+0000","last_undegraded":"2023-11-24T06:24:20.854429+0000","last_fullsized":"2023-11-24T06:24:20.854429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860367+0000","last_change":"2023-11-24T06:24:18.853953+0000","last_active":"2023-11-24T06:24:19.860367+0000","last_peered":"2023-11-24T06:24:19.860367+0000","last_clean":"2023-11-24T06:24:19.860367+0000","last_became_active":"2023-11-24T06:24:18.853698+0000","last_became_peered":"2023-11-24T06:24:18.853698+0000","last_unstale":"2023-11-24T06:24:19.860367+0000","last_undegraded":"2023-11-24T06:24:19.860367+0000","last_fullsized":"2023-11-24T06:24:19.860367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553322+0000","last_change":"2023-11-24T06:24:19.873616+0000","last_active":"2023-11-24T06:24:21.553322+0000","last_peered":"2023-11-24T06:24:21.553322+0000","last_clean":"2023-11-24T06:24:21.553322+0000","last_became_active":"2023-11-24T06:24:19.873308+0000","last_became_peered":"2023-11-24T06:24:19.873308+0000","last_unstale":"2023-11-24T06:24:21.553322+0000","last_undegraded":"2023-11-24T06:24:21.553322+0000","last_fullsized":"2023-11-24T06:24:21.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.886102+0000","last_change":"2023-11-24T06:24:18.863914+0000","last_active":"2023-11-24T06:24:19.886102+0000","last_peered":"2023-11-24T06:24:19.886102+0000","last_clean":"2023-11-24T06:24:19.886102+0000","last_became_active":"2023-11-24T06:24:18.862847+0000","last_became_peered":"2023-11-24T06:24:18.862847+0000","last_unstale":"2023-11-24T06:24:19.886102+0000","last_undegraded":"2023-11-24T06:24:19.886102+0000","last_fullsized":"2023-11-24T06:24:19.886102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854591+0000","last_change":"2023-11-24T06:24:19.865497+0000","last_active":"2023-11-24T06:24:20.854591+0000","last_peered":"2023-11-24T06:24:20.854591+0000","last_clean":"2023-11-24T06:24:20.854591+0000","last_became_active":"2023-11-24T06:24:19.863266+0000","last_became_peered":"2023-11-24T06:24:19.863266+0000","last_unstale":"2023-11-24T06:24:20.854591+0000","last_undegraded":"2023-11-24T06:24:20.854591+0000","last_fullsized":"2023-11-24T06:24:20.854591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:28.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.883170+0000","last_change":"2023-11-24T06:24:18.854874+0000","last_active":"2023-11-24T06:24:19.883170+0000","last_peered":"2023-11-24T06:24:19.883170+0000","last_clean":"2023-11-24T06:24:19.883170+0000","last_became_active":"2023-11-24T06:24:18.854110+0000","last_became_peered":"2023-11-24T06:24:18.854110+0000","last_unstale":"2023-11-24T06:24:19.883170+0000","last_undegraded":"2023-11-24T06:24:19.883170+0000","last_fullsized":"2023-11-24T06:24:19.883170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:38:02.857773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854478+0000","last_change":"2023-11-24T06:24:19.862689+0000","last_active":"2023-11-24T06:24:20.854478+0000","last_peered":"2023-11-24T06:24:20.854478+0000","last_clean":"2023-11-24T06:24:20.854478+0000","last_became_active":"2023-11-24T06:24:19.862254+0000","last_became_peered":"2023-11-24T06:24:19.862254+0000","last_unstale":"2023-11-24T06:24:20.854478+0000","last_undegraded":"2023-11-24T06:24:20.854478+0000","last_fullsized":"2023-11-24T06:24:20.854478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:42.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.860268+0000","last_change":"2023-11-24T06:24:18.862676+0000","last_active":"2023-11-24T06:24:19.860268+0000","last_peered":"2023-11-24T06:24:19.860268+0000","last_clean":"2023-11-24T06:24:19.860268+0000","last_became_active":"2023-11-24T06:24:18.862394+0000","last_became_peered":"2023-11-24T06:24:18.862394+0000","last_unstale":"2023-11-24T06:24:19.860268+0000","last_undegraded":"2023-11-24T06:24:19.860268+0000","last_fullsized":"2023-11-24T06:24:19.860268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:29.750319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054699+0000","last_change":"2023-11-24T06:24:19.876750+0000","last_active":"2023-11-24T06:24:21.054699+0000","last_peered":"2023-11-24T06:24:21.054699+0000","last_clean":"2023-11-24T06:24:21.054699+0000","last_became_active":"2023-11-24T06:24:19.876463+0000","last_became_peered":"2023-11-24T06:24:19.876463+0000","last_unstale":"2023-11-24T06:24:21.054699+0000","last_undegraded":"2023-11-24T06:24:21.054699+0000","last_fullsized":"2023-11-24T06:24:21.054699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.864116+0000","last_change":"2023-11-24T06:24:18.863407+0000","last_active":"2023-11-24T06:24:19.864116+0000","last_peered":"2023-11-24T06:24:19.864116+0000","last_clean":"2023-11-24T06:24:19.864116+0000","last_became_active":"2023-11-24T06:24:18.863107+0000","last_became_peered":"2023-11-24T06:24:18.863107+0000","last_unstale":"2023-11-24T06:24:19.864116+0000","last_undegraded":"2023-11-24T06:24:19.864116+0000","last_fullsized":"2023-11-24T06:24:19.864116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855678+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:20.855678+0000","last_peered":"2023-11-24T06:24:20.855678+0000","last_clean":"2023-11-24T06:24:20.855678+0000","last_became_active":"2023-11-24T06:24:19.866108+0000","last_became_peered":"2023-11-24T06:24:19.866108+0000","last_unstale":"2023-11-24T06:24:20.855678+0000","last_undegraded":"2023-11-24T06:24:20.855678+0000","last_fullsized":"2023-11-24T06:24:20.855678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:03.439397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.852935+0000","last_change":"2023-11-24T06:24:19.865135+0000","last_active":"2023-11-24T06:24:20.852935+0000","last_peered":"2023-11-24T06:24:20.852935+0000","last_clean":"2023-11-24T06:24:20.852935+0000","last_became_active":"2023-11-24T06:24:19.864837+0000","last_became_peered":"2023-11-24T06:24:19.864837+0000","last_unstale":"2023-11-24T06:24:20.852935+0000","last_undegraded":"2023-11-24T06:24:20.852935+0000","last_fullsized":"2023-11-24T06:24:20.852935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857378+0000","last_change":"2023-11-24T06:24:18.862382+0000","last_active":"2023-11-24T06:24:19.857378+0000","last_peered":"2023-11-24T06:24:19.857378+0000","last_clean":"2023-11-24T06:24:19.857378+0000","last_became_active":"2023-11-24T06:24:18.862187+0000","last_became_peered":"2023-11-24T06:24:18.862187+0000","last_unstale":"2023-11-24T06:24:19.857378+0000","last_undegraded":"2023-11-24T06:24:19.857378+0000","last_fullsized":"2023-11-24T06:24:19.857378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904182+0000","last_change":"2023-11-24T06:24:19.871367+0000","last_active":"2023-11-24T06:24:20.904182+0000","last_peered":"2023-11-24T06:24:20.904182+0000","last_clean":"2023-11-24T06:24:20.904182+0000","last_became_active":"2023-11-24T06:24:19.871058+0000","last_became_peered":"2023-11-24T06:24:19.871058+0000","last_unstale":"2023-11-24T06:24:20.904182+0000","last_undegraded":"2023-11-24T06:24:20.904182+0000","last_fullsized":"2023-11-24T06:24:20.904182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:09.507623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.858425+0000","last_change":"2023-11-24T06:24:18.862632+0000","last_active":"2023-11-24T06:24:19.858425+0000","last_peered":"2023-11-24T06:24:19.858425+0000","last_clean":"2023-11-24T06:24:19.858425+0000","last_became_active":"2023-11-24T06:24:18.862317+0000","last_became_peered":"2023-11-24T06:24:18.862317+0000","last_unstale":"2023-11-24T06:24:19.858425+0000","last_undegraded":"2023-11-24T06:24:19.858425+0000","last_fullsized":"2023-11-24T06:24:19.858425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852316+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:24:19.852316+0000","last_peered":"2023-11-24T06:24:19.852316+0000","last_clean":"2023-11-24T06:24:19.852316+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:24:19.852316+0000","last_undegraded":"2023-11-24T06:24:19.852316+0000","last_fullsized":"2023-11-24T06:24:19.852316+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979177+0000","last_change":"2023-11-24T06:24:19.864862+0000","last_active":"2023-11-24T06:24:20.979177+0000","last_peered":"2023-11-24T06:24:20.979177+0000","last_clean":"2023-11-24T06:24:20.979177+0000","last_became_active":"2023-11-24T06:24:19.863541+0000","last_became_peered":"2023-11-24T06:24:19.863541+0000","last_unstale":"2023-11-24T06:24:20.979177+0000","last_undegraded":"2023-11-24T06:24:20.979177+0000","last_fullsized":"2023-11-24T06:24:20.979177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852544+0000","last_change":"2023-11-24T06:24:18.855540+0000","last_active":"2023-11-24T06:24:19.852544+0000","last_peered":"2023-11-24T06:24:19.852544+0000","last_clean":"2023-11-24T06:24:19.852544+0000","last_became_active":"2023-11-24T06:24:18.855238+0000","last_became_peered":"2023-11-24T06:24:18.855238+0000","last_unstale":"2023-11-24T06:24:19.852544+0000","last_undegraded":"2023-11-24T06:24:19.852544+0000","last_fullsized":"2023-11-24T06:24:19.852544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:02.435734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.853979+0000","last_change":"2023-11-24T06:24:18.858903+0000","last_active":"2023-11-24T06:24:19.853979+0000","last_peered":"2023-11-24T06:24:19.853979+0000","last_clean":"2023-11-24T06:24:19.853979+0000","last_became_active":"2023-11-24T06:24:18.857033+0000","last_became_peered":"2023-11-24T06:24:18.857033+0000","last_unstale":"2023-11-24T06:24:19.853979+0000","last_undegraded":"2023-11-24T06:24:19.853979+0000","last_fullsized":"2023-11-24T06:24:19.853979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:48.736229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853074+0000","last_change":"2023-11-24T06:24:19.870498+0000","last_active":"2023-11-24T06:24:20.853074+0000","last_peered":"2023-11-24T06:24:20.853074+0000","last_clean":"2023-11-24T06:24:20.853074+0000","last_became_active":"2023-11-24T06:24:19.870199+0000","last_became_peered":"2023-11-24T06:24:19.870199+0000","last_unstale":"2023-11-24T06:24:20.853074+0000","last_undegraded":"2023-11-24T06:24:20.853074+0000","last_fullsized":"2023-11-24T06:24:20.853074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853198+0000","last_change":"2023-11-24T06:24:18.858061+0000","last_active":"2023-11-24T06:24:20.853198+0000","last_peered":"2023-11-24T06:24:20.853198+0000","last_clean":"2023-11-24T06:24:20.853198+0000","last_became_active":"2023-11-24T06:24:18.857779+0000","last_became_peered":"2023-11-24T06:24:18.857779+0000","last_unstale":"2023-11-24T06:24:20.853198+0000","last_undegraded":"2023-11-24T06:24:20.853198+0000","last_fullsized":"2023-11-24T06:24:20.853198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:15.928175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904362+0000","last_change":"2023-11-24T06:24:19.869468+0000","last_active":"2023-11-24T06:24:20.904362+0000","last_peered":"2023-11-24T06:24:20.904362+0000","last_clean":"2023-11-24T06:24:20.904362+0000","last_became_active":"2023-11-24T06:24:19.869176+0000","last_became_peered":"2023-11-24T06:24:19.869176+0000","last_unstale":"2023-11-24T06:24:20.904362+0000","last_undegraded":"2023-11-24T06:24:20.904362+0000","last_fullsized":"2023-11-24T06:24:20.904362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.880276+0000","last_change":"2023-11-24T06:24:18.862870+0000","last_active":"2023-11-24T06:24:19.880276+0000","last_peered":"2023-11-24T06:24:19.880276+0000","last_clean":"2023-11-24T06:24:19.880276+0000","last_became_active":"2023-11-24T06:24:18.862592+0000","last_became_peered":"2023-11-24T06:24:18.862592+0000","last_unstale":"2023-11-24T06:24:19.880276+0000","last_undegraded":"2023-11-24T06:24:19.880276+0000","last_fullsized":"2023-11-24T06:24:19.880276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979489+0000","last_change":"2023-11-24T06:24:19.868306+0000","last_active":"2023-11-24T06:24:20.979489+0000","last_peered":"2023-11-24T06:24:20.979489+0000","last_clean":"2023-11-24T06:24:20.979489+0000","last_became_active":"2023-11-24T06:24:19.867148+0000","last_became_peered":"2023-11-24T06:24:19.867148+0000","last_unstale":"2023-11-24T06:24:20.979489+0000","last_undegraded":"2023-11-24T06:24:20.979489+0000","last_fullsized":"2023-11-24T06:24:20.979489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:30.736841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.852405+0000","last_change":"2023-11-24T06:24:18.860070+0000","last_active":"2023-11-24T06:24:19.852405+0000","last_peered":"2023-11-24T06:24:19.852405+0000","last_clean":"2023-11-24T06:24:19.852405+0000","last_became_active":"2023-11-24T06:24:18.857288+0000","last_became_peered":"2023-11-24T06:24:18.857288+0000","last_unstale":"2023-11-24T06:24:19.852405+0000","last_undegraded":"2023-11-24T06:24:19.852405+0000","last_fullsized":"2023-11-24T06:24:19.852405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.855451+0000","last_change":"2023-11-24T06:24:19.862616+0000","last_active":"2023-11-24T06:24:20.855451+0000","last_peered":"2023-11-24T06:24:20.855451+0000","last_clean":"2023-11-24T06:24:20.855451+0000","last_became_active":"2023-11-24T06:24:19.862284+0000","last_became_peered":"2023-11-24T06:24:19.862284+0000","last_unstale":"2023-11-24T06:24:20.855451+0000","last_undegraded":"2023-11-24T06:24:20.855451+0000","last_fullsized":"2023-11-24T06:24:20.855451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:46.608575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859914+0000","last_change":"2023-11-24T06:24:18.860580+0000","last_active":"2023-11-24T06:24:19.859914+0000","last_peered":"2023-11-24T06:24:19.859914+0000","last_clean":"2023-11-24T06:24:19.859914+0000","last_became_active":"2023-11-24T06:24:18.860254+0000","last_became_peered":"2023-11-24T06:24:18.860254+0000","last_unstale":"2023-11-24T06:24:19.859914+0000","last_undegraded":"2023-11-24T06:24:19.859914+0000","last_fullsized":"2023-11-24T06:24:19.859914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:04.882895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904102+0000","last_change":"2023-11-24T06:24:19.866929+0000","last_active":"2023-11-24T06:24:20.904102+0000","last_peered":"2023-11-24T06:24:20.904102+0000","last_clean":"2023-11-24T06:24:20.904102+0000","last_became_active":"2023-11-24T06:24:19.866522+0000","last_became_peered":"2023-11-24T06:24:19.866522+0000","last_unstale":"2023-11-24T06:24:20.904102+0000","last_undegraded":"2023-11-24T06:24:20.904102+0000","last_fullsized":"2023-11-24T06:24:20.904102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.859599+0000","last_change":"2023-11-24T06:24:18.862071+0000","last_active":"2023-11-24T06:24:19.859599+0000","last_peered":"2023-11-24T06:24:19.859599+0000","last_clean":"2023-11-24T06:24:19.859599+0000","last_became_active":"2023-11-24T06:24:18.861759+0000","last_became_peered":"2023-11-24T06:24:18.861759+0000","last_unstale":"2023-11-24T06:24:19.859599+0000","last_undegraded":"2023-11-24T06:24:19.859599+0000","last_fullsized":"2023-11-24T06:24:19.859599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:57:45.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904434+0000","last_change":"2023-11-24T06:24:19.870623+0000","last_active":"2023-11-24T06:24:20.904434+0000","last_peered":"2023-11-24T06:24:20.904434+0000","last_clean":"2023-11-24T06:24:20.904434+0000","last_became_active":"2023-11-24T06:24:19.869946+0000","last_became_peered":"2023-11-24T06:24:19.869946+0000","last_unstale":"2023-11-24T06:24:20.904434+0000","last_undegraded":"2023-11-24T06:24:20.904434+0000","last_fullsized":"2023-11-24T06:24:20.904434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:13.582566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.879779+0000","last_change":"2023-11-24T06:24:18.855948+0000","last_active":"2023-11-24T06:24:19.879779+0000","last_peered":"2023-11-24T06:24:19.879779+0000","last_clean":"2023-11-24T06:24:19.879779+0000","last_became_active":"2023-11-24T06:24:18.855185+0000","last_became_peered":"2023-11-24T06:24:18.855185+0000","last_unstale":"2023-11-24T06:24:19.879779+0000","last_undegraded":"2023-11-24T06:24:19.879779+0000","last_fullsized":"2023-11-24T06:24:19.879779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:40:08.503960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553085+0000","last_change":"2023-11-24T06:24:19.866394+0000","last_active":"2023-11-24T06:24:21.553085+0000","last_peered":"2023-11-24T06:24:21.553085+0000","last_clean":"2023-11-24T06:24:21.553085+0000","last_became_active":"2023-11-24T06:24:19.866115+0000","last_became_peered":"2023-11-24T06:24:19.866115+0000","last_unstale":"2023-11-24T06:24:21.553085+0000","last_undegraded":"2023-11-24T06:24:21.553085+0000","last_fullsized":"2023-11-24T06:24:21.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:09.353866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.857112+0000","last_change":"2023-11-24T06:24:18.864397+0000","last_active":"2023-11-24T06:24:19.857112+0000","last_peered":"2023-11-24T06:24:19.857112+0000","last_clean":"2023-11-24T06:24:19.857112+0000","last_became_active":"2023-11-24T06:24:18.864151+0000","last_became_peered":"2023-11-24T06:24:18.864151+0000","last_unstale":"2023-11-24T06:24:19.857112+0000","last_undegraded":"2023-11-24T06:24:19.857112+0000","last_fullsized":"2023-11-24T06:24:19.857112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:27.842094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854430+0000","last_change":"2023-11-24T06:24:19.867560+0000","last_active":"2023-11-24T06:24:20.854430+0000","last_peered":"2023-11-24T06:24:20.854430+0000","last_clean":"2023-11-24T06:24:20.854430+0000","last_became_active":"2023-11-24T06:24:19.867277+0000","last_became_peered":"2023-11-24T06:24:19.867277+0000","last_unstale":"2023-11-24T06:24:20.854430+0000","last_undegraded":"2023-11-24T06:24:20.854430+0000","last_fullsized":"2023-11-24T06:24:20.854430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.885158+0000","last_change":"2023-11-24T06:24:18.863072+0000","last_active":"2023-11-24T06:24:19.885158+0000","last_peered":"2023-11-24T06:24:19.885158+0000","last_clean":"2023-11-24T06:24:19.885158+0000","last_became_active":"2023-11-24T06:24:18.862782+0000","last_became_peered":"2023-11-24T06:24:18.862782+0000","last_unstale":"2023-11-24T06:24:19.885158+0000","last_undegraded":"2023-11-24T06:24:19.885158+0000","last_fullsized":"2023-11-24T06:24:19.885158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:32.480639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054548+0000","last_change":"2023-11-24T06:24:19.875426+0000","last_active":"2023-11-24T06:24:21.054548+0000","last_peered":"2023-11-24T06:24:21.054548+0000","last_clean":"2023-11-24T06:24:21.054548+0000","last_became_active":"2023-11-24T06:24:19.875167+0000","last_became_peered":"2023-11-24T06:24:19.875167+0000","last_unstale":"2023-11-24T06:24:21.054548+0000","last_undegraded":"2023-11-24T06:24:21.054548+0000","last_fullsized":"2023-11-24T06:24:21.054548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:59.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:24:19.884010+0000","last_change":"2023-11-24T06:24:18.857998+0000","last_active":"2023-11-24T06:24:19.884010+0000","last_peered":"2023-11-24T06:24:19.884010+0000","last_clean":"2023-11-24T06:24:19.884010+0000","last_became_active":"2023-11-24T06:24:18.857716+0000","last_became_peered":"2023-11-24T06:24:18.857716+0000","last_unstale":"2023-11-24T06:24:19.884010+0000","last_undegraded":"2023-11-24T06:24:19.884010+0000","last_fullsized":"2023-11-24T06:24:19.884010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:17.831105+0000","last_clean_scrub_stamp":"2023-11-24T06:24:17.831105+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:54:50.045740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.854899+0000","last_change":"2023-11-24T06:24:19.866835+0000","last_active":"2023-11-24T06:24:20.854899+0000","last_peered":"2023-11-24T06:24:20.854899+0000","last_clean":"2023-11-24T06:24:20.854899+0000","last_became_active":"2023-11-24T06:24:19.866476+0000","last_became_peered":"2023-11-24T06:24:19.866476+0000","last_unstale":"2023-11-24T06:24:20.854899+0000","last_undegraded":"2023-11-24T06:24:20.854899+0000","last_fullsized":"2023-11-24T06:24:20.854899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:20.926427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553269+0000","last_change":"2023-11-24T06:24:19.873151+0000","last_active":"2023-11-24T06:24:21.553269+0000","last_peered":"2023-11-24T06:24:21.553269+0000","last_clean":"2023-11-24T06:24:21.553269+0000","last_became_active":"2023-11-24T06:24:19.872848+0000","last_became_peered":"2023-11-24T06:24:19.872848+0000","last_unstale":"2023-11-24T06:24:21.553269+0000","last_undegraded":"2023-11-24T06:24:21.553269+0000","last_fullsized":"2023-11-24T06:24:21.553269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:52.890601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553077+0000","last_change":"2023-11-24T06:24:19.866025+0000","last_active":"2023-11-24T06:24:21.553077+0000","last_peered":"2023-11-24T06:24:21.553077+0000","last_clean":"2023-11-24T06:24:21.553077+0000","last_became_active":"2023-11-24T06:24:19.865501+0000","last_became_peered":"2023-11-24T06:24:19.865501+0000","last_unstale":"2023-11-24T06:24:21.553077+0000","last_undegraded":"2023-11-24T06:24:21.553077+0000","last_fullsized":"2023-11-24T06:24:21.553077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:39.781400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054789+0000","last_change":"2023-11-24T06:24:19.876131+0000","last_active":"2023-11-24T06:24:21.054789+0000","last_peered":"2023-11-24T06:24:21.054789+0000","last_clean":"2023-11-24T06:24:21.054789+0000","last_became_active":"2023-11-24T06:24:19.875876+0000","last_became_peered":"2023-11-24T06:24:19.875876+0000","last_unstale":"2023-11-24T06:24:21.054789+0000","last_undegraded":"2023-11-24T06:24:21.054789+0000","last_fullsized":"2023-11-24T06:24:21.054789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:24.245884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979358+0000","last_change":"2023-11-24T06:24:19.865439+0000","last_active":"2023-11-24T06:24:20.979358+0000","last_peered":"2023-11-24T06:24:20.979358+0000","last_clean":"2023-11-24T06:24:20.979358+0000","last_became_active":"2023-11-24T06:24:19.865124+0000","last_became_peered":"2023-11-24T06:24:19.865124+0000","last_unstale":"2023-11-24T06:24:20.979358+0000","last_undegraded":"2023-11-24T06:24:20.979358+0000","last_fullsized":"2023-11-24T06:24:20.979358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:30.091481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904352+0000","last_change":"2023-11-24T06:24:19.865010+0000","last_active":"2023-11-24T06:24:20.904352+0000","last_peered":"2023-11-24T06:24:20.904352+0000","last_clean":"2023-11-24T06:24:20.904352+0000","last_became_active":"2023-11-24T06:24:19.864730+0000","last_became_peered":"2023-11-24T06:24:19.864730+0000","last_unstale":"2023-11-24T06:24:20.904352+0000","last_undegraded":"2023-11-24T06:24:20.904352+0000","last_fullsized":"2023-11-24T06:24:20.904352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.853108+0000","last_change":"2023-11-24T06:24:19.873933+0000","last_active":"2023-11-24T06:24:20.853108+0000","last_peered":"2023-11-24T06:24:20.853108+0000","last_clean":"2023-11-24T06:24:20.853108+0000","last_became_active":"2023-11-24T06:24:19.873679+0000","last_became_peered":"2023-11-24T06:24:19.873679+0000","last_unstale":"2023-11-24T06:24:20.853108+0000","last_undegraded":"2023-11-24T06:24:20.853108+0000","last_fullsized":"2023-11-24T06:24:20.853108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553458+0000","last_change":"2023-11-24T06:24:19.873987+0000","last_active":"2023-11-24T06:24:21.553458+0000","last_peered":"2023-11-24T06:24:21.553458+0000","last_clean":"2023-11-24T06:24:21.553458+0000","last_became_active":"2023-11-24T06:24:19.871835+0000","last_became_peered":"2023-11-24T06:24:19.871835+0000","last_unstale":"2023-11-24T06:24:21.553458+0000","last_undegraded":"2023-11-24T06:24:21.553458+0000","last_fullsized":"2023-11-24T06:24:21.553458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904255+0000","last_change":"2023-11-24T06:24:19.863300+0000","last_active":"2023-11-24T06:24:20.904255+0000","last_peered":"2023-11-24T06:24:20.904255+0000","last_clean":"2023-11-24T06:24:20.904255+0000","last_became_active":"2023-11-24T06:24:19.861918+0000","last_became_peered":"2023-11-24T06:24:19.861918+0000","last_unstale":"2023-11-24T06:24:20.904255+0000","last_undegraded":"2023-11-24T06:24:20.904255+0000","last_fullsized":"2023-11-24T06:24:20.904255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054464+0000","last_change":"2023-11-24T06:24:19.878153+0000","last_active":"2023-11-24T06:24:21.054464+0000","last_peered":"2023-11-24T06:24:21.054464+0000","last_clean":"2023-11-24T06:24:21.054464+0000","last_became_active":"2023-11-24T06:24:19.876482+0000","last_became_peered":"2023-11-24T06:24:19.876482+0000","last_unstale":"2023-11-24T06:24:21.054464+0000","last_undegraded":"2023-11-24T06:24:21.054464+0000","last_fullsized":"2023-11-24T06:24:21.054464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.596949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.904064+0000","last_change":"2023-11-24T06:24:19.862468+0000","last_active":"2023-11-24T06:24:20.904064+0000","last_peered":"2023-11-24T06:24:20.904064+0000","last_clean":"2023-11-24T06:24:20.904064+0000","last_became_active":"2023-11-24T06:24:19.862178+0000","last_became_peered":"2023-11-24T06:24:19.862178+0000","last_unstale":"2023-11-24T06:24:20.904064+0000","last_undegraded":"2023-11-24T06:24:20.904064+0000","last_fullsized":"2023-11-24T06:24:20.904064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:26.495227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.054544+0000","last_change":"2023-11-24T06:24:19.871577+0000","last_active":"2023-11-24T06:24:21.054544+0000","last_peered":"2023-11-24T06:24:21.054544+0000","last_clean":"2023-11-24T06:24:21.054544+0000","last_became_active":"2023-11-24T06:24:19.871279+0000","last_became_peered":"2023-11-24T06:24:19.871279+0000","last_unstale":"2023-11-24T06:24:21.054544+0000","last_undegraded":"2023-11-24T06:24:21.054544+0000","last_fullsized":"2023-11-24T06:24:21.054544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:10.271512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:20.979353+0000","last_change":"2023-11-24T06:24:19.872608+0000","last_active":"2023-11-24T06:24:20.979353+0000","last_peered":"2023-11-24T06:24:20.979353+0000","last_clean":"2023-11-24T06:24:20.979353+0000","last_became_active":"2023-11-24T06:24:19.872343+0000","last_became_peered":"2023-11-24T06:24:19.872343+0000","last_unstale":"2023-11-24T06:24:20.979353+0000","last_undegraded":"2023-11-24T06:24:20.979353+0000","last_fullsized":"2023-11-24T06:24:20.979353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:24:21.553301+0000","last_change":"2023-11-24T06:24:19.868776+0000","last_active":"2023-11-24T06:24:21.553301+0000","last_peered":"2023-11-24T06:24:21.553301+0000","last_clean":"2023-11-24T06:24:21.553301+0000","last_became_active":"2023-11-24T06:24:19.868335+0000","last_became_peered":"2023-11-24T06:24:19.868335+0000","last_unstale":"2023-11-24T06:24:21.553301+0000","last_undegraded":"2023-11-24T06:24:21.553301+0000","last_fullsized":"2023-11-24T06:24:21.553301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:18.834768+0000","last_clean_scrub_stamp":"2023-11-24T06:24:18.834768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:20.738668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:24:33.558 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:24:33.558 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:24:33.558 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:24:33.559 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:24:33.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.710+0000 7f689d405700 1 Processor -- start 2023-11-24T06:24:33.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.710+0000 7f689d405700 1 -- start start 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f689d405700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f689815e080 0x7f68980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f689d405700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689806c710 0x7f68980afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f689d405700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980ae430 con 0x7f68980af4d0 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f689d405700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980af0b0 con 0x7f689815e080 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f689d405700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b05a0 con 0x7f689806c710 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f689815e080 0x7f68980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f68967fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f68980af4d0 0x7f689806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54018/0 (socket says 172.21.15.114:54018) 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f68967fc700 1 -- 172.21.15.114:0/818820415 learned_addr learned my addr 172.21.15.114:0/818820415 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f68977fe700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689806c710 0x7f68980afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6895ffb700 1 -- 172.21.15.114:0/818820415 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750211043 0 0) 0x7f68980ae430 con 0x7f68980af4d0 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6895ffb700 1 -- 172.21.15.114:0/818820415 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68800035b0 con 0x7f68980af4d0 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 -- 172.21.15.114:0/818820415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689806c710 msgr2=0x7f68980afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689806c710 0x7f68980afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 -- 172.21.15.114:0/818820415 >> v1:172.21.15.114:6789/0 conn(0x7f68980af4d0 legacy=0x7f689806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 -- 172.21.15.114:0/818820415 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980ae710 con 0x7f689815e080 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6896ffd700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689815e080 0x7f68980b5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6888009870 tx=0x7f6888009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680b56a7e0a0b788 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6895ffb700 1 -- 172.21.15.114:0/818820415 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888011830 con 0x7f689815e080 2023-11-24T06:24:33.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6895ffb700 1 -- 172.21.15.114:0/818820415 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888011990 con 0x7f689815e080 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.711+0000 7f6895ffb700 1 -- 172.21.15.114:0/818820415 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888011d20 con 0x7f689815e080 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 -- 172.21.15.114:0/818820415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689815e080 msgr2=0x7f68980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689815e080 0x7f68980b5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6888009870 tx=0x7f6888009b80 comp rx=0 tx=0).stop 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 -- 172.21.15.114:0/818820415 shutdown_connections 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689806c710 0x7f68980afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 --2- 172.21.15.114:0/818820415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689815e080 0x7f68980b5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 -- 172.21.15.114:0/818820415 >> 172.21.15.114:0/818820415 conn(0x7f6898071b90 msgr2=0x7f6898071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 -- 172.21.15.114:0/818820415 shutdown_connections 2023-11-24T06:24:33.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.712+0000 7f689d405700 1 -- 172.21.15.114:0/818820415 wait complete. 2023-11-24T06:24:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.713+0000 7f689d405700 1 Processor -- start 2023-11-24T06:24:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.713+0000 7f689d405700 1 -- start start 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689806c710 0x7f68980cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689815e080 0x7f6898157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689815ef10 con 0x7f68980af4d0 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898159ef0 con 0x7f689806c710 2023-11-24T06:24:33.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f689d405700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689815a1d0 con 0x7f689815e080 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38316/0 (socket says 172.21.15.114:38316) 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 -- 172.21.15.114:0/2099153690 learned_addr learned my addr 172.21.15.114:0/2099153690 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f6896ffd700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689806c710 0x7f68980cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68977fe700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689815e080 0x7f6898157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 -- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689815e080 msgr2=0x7f6898157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689815e080 0x7f6898157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 -- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689806c710 msgr2=0x7f68980cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689806c710 0x7f68980cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.714+0000 7f68967fc700 1 -- 172.21.15.114:0/2099153690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880094a0 con 0x7f68980af4d0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f68967fc700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f688c005910 tx=0x7f688c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c005d70 con 0x7f68980af4d0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c0070d0 con 0x7f68980af4d0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689815a4b0 con 0x7f68980af4d0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c007460 con 0x7f68980af4d0 2023-11-24T06:24:33.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.715+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6898155010 con 0x7f68980af4d0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.716+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980753f0 con 0x7f68980af4d0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.716+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f688c0075c0 con 0x7f68980af4d0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.716+0000 7f687f7fe700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 0x7f688406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.717+0000 7f6896ffd700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 0x7f688406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.717+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f688c088b90 con 0x7f68980af4d0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.717+0000 7f6896ffd700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 0x7f688406d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f68980687d0 tx=0x7f6888008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.721+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f688c053c40 con 0x7f68980af4d0 2023-11-24T06:24:33.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.956+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f68980690c0 con 0x7f68980af4d0 2023-11-24T06:24:33.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.957+0000 7f687f7fe700 1 -- 172.21.15.114:0/2099153690 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f688c017040 con 0x7f68980af4d0 2023-11-24T06:24:33.961 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 msgr2=0x7f688406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 0x7f688406d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f68980687d0 tx=0x7f6888008030 comp rx=0 tx=0).stop 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 msgr2=0x7f6898159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f688c005910 tx=0x7f688c00ca90 comp rx=0 tx=0).stop 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 shutdown_connections 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f688406aed0 0x7f688406d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f689815e080 0x7f6898157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689806c710 0x7f68980cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 --2- 172.21.15.114:0/2099153690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68980af4d0 0x7f6898159530 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:33.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 >> 172.21.15.114:0/2099153690 conn(0x7f6898071b90 msgr2=0x7f68980b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:33.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 shutdown_connections 2023-11-24T06:24:33.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:33.959+0000 7f689d405700 1 -- 172.21.15.114:0/2099153690 wait complete. 2023-11-24T06:24:33.974 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:24:33.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:34.096 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.092+0000 7ff6cdd3c700 1 Processor -- start 2023-11-24T06:24:34.096 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.092+0000 7ff6cdd3c700 1 -- start start 2023-11-24T06:24:34.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6cdd3c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6c80adf00 0x7ff6c806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6cdd3c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c80af950 con 0x7ff6c806c540 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6cdd3c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c80afc30 con 0x7ff6c81446a0 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6cdd3c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c806c9f0 con 0x7ff6c80adf00 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c7fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff6c806c540 0x7ff6c80b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54034/0 (socket says 172.21.15.114:54034) 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c7fff700 1 -- 172.21.15.114:0/2900165824 learned_addr learned my addr 172.21.15.114:0/2900165824 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204122853 0 0) 0x7ff6c80af950 con 0x7ff6c806c540 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c6ffd700 1 --2- 172.21.15.114:0/2900165824 >> v2:172.21.15.172:3300/0 conn(0x7ff6c80adf00 0x7ff6c806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.105 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b00035b0 con 0x7ff6c806c540 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 429160427 0 0) 0x7ff6b00035b0 con 0x7ff6c806c540 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 >> v2:172.21.15.172:3300/0 conn(0x7ff6c80adf00 msgr2=0x7ff6c806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 --2- 172.21.15.114:0/2900165824 >> v2:172.21.15.172:3300/0 conn(0x7ff6c80adf00 0x7ff6c806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 >> v1:172.21.15.172:6789/0 conn(0x7ff6c81446a0 legacy=0x7ff6c80b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c80aff10 con 0x7ff6c806c540 2023-11-24T06:24:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7ff6c00019f0 con 0x7ff6c806c540 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 >> v1:172.21.15.114:6789/0 conn(0x7ff6c806c540 legacy=0x7ff6c80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 0x7ff6b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6b0004850 0x7ff6b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0005210 0x7ff6b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c806c9f0 con 0x7ff6b0003a00 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b000c0b0 con 0x7ff6b0005210 2023-11-24T06:24:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b000c270 con 0x7ff6b0004850 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 0x7ff6b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c6ffd700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6b0004850 0x7ff6b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.094+0000 7ff6c7fff700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0005210 0x7ff6b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 -- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6b0004850 msgr2=0x7ff6b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6b0004850 0x7ff6b0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 -- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0005210 msgr2=0x7ff6b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0005210 0x7ff6b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 -- 172.21.15.114:0/2900165824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00035b0 con 0x7ff6b0003a00 2023-11-24T06:24:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c7fff700 1 -- 172.21.15.114:0/2900165824 reap_dead start 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 0x7ff6b0003e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0017c0 tx=0x7ff6b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae651835249ede65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0052d0 con 0x7ff6b0003a00 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc005450 con 0x7ff6b0003a00 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6c67fc700 1 -- 172.21.15.114:0/2900165824 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0038c0 con 0x7ff6b0003a00 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 msgr2=0x7ff6b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 0x7ff6b0003e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0017c0 tx=0x7ff6b0005880 comp rx=0 tx=0).stop 2023-11-24T06:24:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/2900165824 shutdown_connections 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/2900165824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6b0003a00 0x7ff6b0003e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/2900165824 >> 172.21.15.114:0/2900165824 conn(0x7ff6c8071b90 msgr2=0x7ff6c8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/2900165824 shutdown_connections 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.095+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/2900165824 wait complete. 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.097+0000 7ff6cdd3c700 1 Processor -- start 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.097+0000 7ff6cdd3c700 1 -- start start 2023-11-24T06:24:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0003a00 0x7ff6c8150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6c814ced0 0x7ff6c814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c814b3b0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8150d90 con 0x7ff6b0003a00 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6cdd3c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c814d890 con 0x7ff6c814ced0 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38336/0 (socket says 172.21.15.114:38336) 2023-11-24T06:24:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 -- 172.21.15.114:0/3361652826 learned_addr learned my addr 172.21.15.114:0/3361652826 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c7fff700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6c814ced0 0x7ff6c814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0003a00 0x7ff6c8150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 -- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6c814ced0 msgr2=0x7ff6c814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6c814ced0 0x7ff6c814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 -- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0003a00 msgr2=0x7ff6c8150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0003a00 0x7ff6c8150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.098+0000 7ff6c6ffd700 1 -- 172.21.15.114:0/3361652826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c80cfcd0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6c6ffd700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8011680 tx=0x7ff6b8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004440 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b80045a0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c80cf710 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80038c0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.099+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c814afd0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.100+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c80753f0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.100+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6b8003a20 con 0x7ff6c80cecb0 2023-11-24T06:24:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.100+0000 7ff6affff700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 0x7ff6b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.100+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6b80899b0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.100+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 0x7ff6b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.101+0000 7ff6c77fe700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 0x7ff6b406d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc015040 tx=0x7ff6bc009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.105+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6b8054a70 con 0x7ff6c80cecb0 2023-11-24T06:24:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.297+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6c814b4f0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.298+0000 7ff6affff700 1 -- 172.21.15.114:0/3361652826 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff6c814b4f0 con 0x7ff6c80cecb0 2023-11-24T06:24:34.302 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:34.302 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 msgr2=0x7ff6b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 0x7ff6b406d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc015040 tx=0x7ff6bc009030 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 msgr2=0x7ff6c80cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8011680 tx=0x7ff6b8011a40 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 shutdown_connections 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff6b406af00 0x7ff6b406d3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6c814ced0 0x7ff6c814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6b0003a00 0x7ff6c8150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 --2- 172.21.15.114:0/3361652826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6c80cecb0 0x7ff6c80cf130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 >> 172.21.15.114:0/3361652826 conn(0x7ff6c8071b90 msgr2=0x7ff6c806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 shutdown_connections 2023-11-24T06:24:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.300+0000 7ff6cdd3c700 1 -- 172.21.15.114:0/3361652826 wait complete. 2023-11-24T06:24:34.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:34.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.437+0000 7f49ffc17700 1 Processor -- start 2023-11-24T06:24:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.437+0000 7f49ffc17700 1 -- start start 2023-11-24T06:24:34.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49ffc17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49ffc17700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f80746a0 0x7f49f806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49ffc17700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f80b3c60 con 0x7f49f80afc90 2023-11-24T06:24:34.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49ffc17700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f80b4500 con 0x7f49f80746a0 2023-11-24T06:24:34.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49ffc17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80b58c0 con 0x7f49f815e160 2023-11-24T06:24:34.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49fe1b4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f80746a0 0x7f49f806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.438+0000 7f49fd9b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fd1b2700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f49f80afc90 0x7f49f8073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47646/0 (socket says 172.21.15.114:47646) 2023-11-24T06:24:34.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f80746a0 0x7f49f806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:34004/0 (socket says 172.21.15.114:34004) 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fd1b2700 1 -- 172.21.15.114:0/85539975 learned_addr learned my addr 172.21.15.114:0/85539975 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 -- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 msgr2=0x7f49f80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 --2- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 -- 172.21.15.114:0/85539975 >> v1:172.21.15.172:6789/0 conn(0x7f49f80afc90 legacy=0x7f49f8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 -- 172.21.15.114:0/85539975 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f806bdc0 con 0x7f49f80746a0 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fc9b1700 1 -- 172.21.15.114:0/85539975 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931688174 0 0) 0x7f49f80b3c60 con 0x7f49f80afc90 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fe1b4700 1 --2- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80746a0 0x7f49f806d930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009d90 tx=0x7f49f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be22fb4cdcbc793 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fc9b1700 1 -- 172.21.15.114:0/85539975 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f000e070 con 0x7f49f80746a0 2023-11-24T06:24:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fc9b1700 1 -- 172.21.15.114:0/85539975 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000bb20 con 0x7f49f80746a0 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.439+0000 7f49fc9b1700 1 -- 172.21.15.114:0/85539975 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0012430 con 0x7f49f80746a0 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 -- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80746a0 msgr2=0x7f49f806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 --2- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80746a0 0x7f49f806d930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009d90 tx=0x7f49f000b330 comp rx=0 tx=0).stop 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 -- 172.21.15.114:0/85539975 shutdown_connections 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 --2- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80746a0 0x7f49f806d930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 --2- 172.21.15.114:0/85539975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 -- 172.21.15.114:0/85539975 >> 172.21.15.114:0/85539975 conn(0x7f49f80d0bf0 msgr2=0x7f49f80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 -- 172.21.15.114:0/85539975 shutdown_connections 2023-11-24T06:24:34.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.440+0000 7f49ffc17700 1 -- 172.21.15.114:0/85539975 wait complete. 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.441+0000 7f49ffc17700 1 Processor -- start 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.441+0000 7f49ffc17700 1 -- start start 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80afc90 0x7f49f8158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8160270 con 0x7f49f80746a0 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815f9d0 con 0x7f49f80afc90 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815fcb0 con 0x7f49f815e160 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38348/0 (socket says 172.21.15.114:38348) 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 -- 172.21.15.114:0/3371511076 learned_addr learned my addr 172.21.15.114:0/3371511076 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd1b2700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80afc90 0x7f49f8158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fe1b4700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 -- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 msgr2=0x7f49f80cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 -- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80afc90 msgr2=0x7f49f8158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80afc90 0x7f49f8158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 -- 172.21.15.114:0/3371511076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009a70 con 0x7f49f80746a0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49fd9b3700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e870 tx=0x7f49e800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8009590 con 0x7f49f80746a0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8013680 con 0x7f49f80746a0 2023-11-24T06:24:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f815ff90 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e80139f0 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.442+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f8158dd0 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.444+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f80c5bd0 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.444+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49e8013b50 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.444+0000 7f49ee7fc700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 0x7f49dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.444+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49e8089360 con 0x7f49f80746a0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.444+0000 7f49fd1b2700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 0x7f49dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.445+0000 7f49fd1b2700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 0x7f49dc06d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49f815be40 tx=0x7f49f4007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.449+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49f80c5bd0 con 0x7f49f80746a0 2023-11-24T06:24:34.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.636+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49f80c5bd0 con 0x7f49f80746a0 2023-11-24T06:24:34.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.637+0000 7f49ee7fc700 1 -- 172.21.15.114:0/3371511076 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49f80c5bd0 con 0x7f49f80746a0 2023-11-24T06:24:34.640 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:34.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 msgr2=0x7f49dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 0x7f49dc06d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49f815be40 tx=0x7f49f4007de0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 msgr2=0x7f49f815f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e870 tx=0x7f49e800eb80 comp rx=0 tx=0).stop 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 shutdown_connections 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49dc06aed0 0x7f49dc06d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49f815e160 0x7f49f80cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49f80746a0 0x7f49f815f180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 --2- 172.21.15.114:0/3371511076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f80afc90 0x7f49f8158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 >> 172.21.15.114:0/3371511076 conn(0x7f49f80d0bf0 msgr2=0x7f49f815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 shutdown_connections 2023-11-24T06:24:34.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.639+0000 7f49ffc17700 1 -- 172.21.15.114:0/3371511076 wait complete. 2023-11-24T06:24:34.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.774+0000 7fe9f4993700 1 Processor -- start 2023-11-24T06:24:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.774+0000 7fe9f4993700 1 -- start start 2023-11-24T06:24:34.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9f4993700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9f00746a0 0x7fe9f00add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9f4993700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f006b080 con 0x7fe9f00cf610 2023-11-24T06:24:34.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9f4993700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f015c290 con 0x7fe9f015f9e0 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9f4993700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf9f0 con 0x7fe9f00746a0 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ee59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe9f00cf610 0x7fe9f00b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54064/0 (socket says 172.21.15.114:54064) 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ee59c700 1 -- 172.21.15.114:0/3602209520 learned_addr learned my addr 172.21.15.114:0/3602209520 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146964490 0 0) 0x7fe9f006b080 con 0x7fe9f00cf610 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d80035b0 con 0x7fe9f00cf610 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3602209520 >> v2:172.21.15.172:3300/0 conn(0x7fe9f00746a0 0x7fe9f00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368786426 0 0) 0x7fe9d80035b0 con 0x7fe9f00cf610 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 >> v2:172.21.15.172:3300/0 conn(0x7fe9f00746a0 msgr2=0x7fe9f00add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 --2- 172.21.15.114:0/3602209520 >> v2:172.21.15.172:3300/0 conn(0x7fe9f00746a0 0x7fe9f00add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 >> v1:172.21.15.172:6789/0 conn(0x7fe9f015f9e0 legacy=0x7fe9f0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f015ff80 con 0x7fe9f00cf610 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fe9e0001670 con 0x7fe9f00cf610 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.775+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 >> v1:172.21.15.114:6789/0 conn(0x7fe9f00cf610 legacy=0x7fe9f00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 0x7fe9d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d8004850 0x7fe9d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8005210 0x7fe9d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf9f0 con 0x7fe9d8003a00 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d800c0e0 con 0x7fe9d8004850 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d800c2a0 con 0x7fe9d8005210 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3602209520 >> v2:172.21.15.172:3300/0 conn(0x7fe9f00746a0 0x7fe9f00add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:34.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 0x7fe9d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8005210 0x7fe9d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 -- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8005210 msgr2=0x7fe9d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8005210 0x7fe9d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 -- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d8004850 msgr2=0x7fe9d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9edd9b700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d8004850 0x7fe9d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d8004850 0x7fe9d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 -- 172.21.15.114:0/3602209520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80035b0 con 0x7fe9d8003a00 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8005210 0x7fe9d800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9edd9b700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d8004850 0x7fe9d8004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9eed9d700 1 -- 172.21.15.114:0/3602209520 reap_dead start 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 0x7fe9d8003de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c2f0 tx=0x7fe9d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2b3ec61ba12d68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000ce30 con 0x7fe9d8003a00 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e0005960 con 0x7fe9d8003a00 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.776+0000 7fe9ed59a700 1 -- 172.21.15.114:0/3602209520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0005d10 con 0x7fe9d8003a00 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 -- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 msgr2=0x7fe9d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 0x7fe9d8003de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c2f0 tx=0x7fe9d8005880 comp rx=0 tx=0).stop 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 -- 172.21.15.114:0/3602209520 shutdown_connections 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3602209520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9d8003a00 0x7fe9d8003de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 -- 172.21.15.114:0/3602209520 >> 172.21.15.114:0/3602209520 conn(0x7fe9f00d0bf0 msgr2=0x7fe9f00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 -- 172.21.15.114:0/3602209520 shutdown_connections 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.777+0000 7fe9f4993700 1 -- 172.21.15.114:0/3602209520 wait complete. 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.778+0000 7fe9f4993700 1 Processor -- start 2023-11-24T06:24:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.778+0000 7fe9f4993700 1 -- start start 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8003a00 0x7fe9f0157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b16d0 0x7fe9f0153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01560f0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01563d0 con 0x7fe9f00b16d0 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9f4993700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01566b0 con 0x7fe9d8003a00 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38372/0 (socket says 172.21.15.114:38372) 2023-11-24T06:24:34.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 -- 172.21.15.114:0/3665325144 learned_addr learned my addr 172.21.15.114:0/3665325144 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8003a00 0x7fe9f0157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9edd9b700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b16d0 0x7fe9f0153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 -- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8003a00 msgr2=0x7fe9f0157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8003a00 0x7fe9f0157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 -- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b16d0 msgr2=0x7fe9f0153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b16d0 0x7fe9f0153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 -- 172.21.15.114:0/3665325144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0156990 con 0x7fe9f00b1b50 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9eed9d700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00d0380 tx=0x7fe9e8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.779+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8007b80 con 0x7fe9f00b1b50 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.780+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8007ce0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.780+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8014640 con 0x7fe9f00b1b50 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.780+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f0156c70 con 0x7fe9f00b1b50 2023-11-24T06:24:34.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.780+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f01576d0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.781+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f00cf1b0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.781+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9e801d3b0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.781+0000 7fe9deffd700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 0x7fe9d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.781+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9e8089060 con 0x7fe9f00b1b50 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.781+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 0x7fe9d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.782+0000 7fe9ee59c700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 0x7fe9d006d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015040 tx=0x7fe9e0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.786+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9e8054110 con 0x7fe9f00b1b50 2023-11-24T06:24:34.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.981+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9f0068fb0 con 0x7fe9f00b1b50 2023-11-24T06:24:34.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.981+0000 7fe9deffd700 1 -- 172.21.15.114:0/3665325144 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9e8057730 con 0x7fe9f00b1b50 2023-11-24T06:24:34.985 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 msgr2=0x7fe9d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 0x7fe9d006d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015040 tx=0x7fe9e0007000 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 msgr2=0x7fe9f0150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00d0380 tx=0x7fe9e8004970 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 shutdown_connections 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe9d006af00 0x7fe9d006d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9f00b1b50 0x7fe9f0150fd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9d8003a00 0x7fe9f0157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 --2- 172.21.15.114:0/3665325144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b16d0 0x7fe9f0153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:34.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 >> 172.21.15.114:0/3665325144 conn(0x7fe9f00d0bf0 msgr2=0x7fe9f00b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:34.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 shutdown_connections 2023-11-24T06:24:34.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:34.983+0000 7fe9f4993700 1 -- 172.21.15.114:0/3665325144 wait complete. 2023-11-24T06:24:34.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.119+0000 7f4605ef4700 1 Processor -- start 2023-11-24T06:24:35.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.119+0000 7f4605ef4700 1 -- start start 2023-11-24T06:24:35.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f4605ef4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46001587f0 0x7f460006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f4605ef4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f46000b4370 0x7f460015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f4605ef4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460006b530 con 0x7f460015b370 2023-11-24T06:24:35.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f4605ef4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46000ae5e0 con 0x7f46000b4370 2023-11-24T06:24:35.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f4605ef4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460015be30 con 0x7f46001587f0 2023-11-24T06:24:35.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f46000b4370 0x7f460015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f46000b4370 0x7f460015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38380/0 (socket says 172.21.15.114:38380) 2023-11-24T06:24:35.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 -- 172.21.15.114:0/2071387431 learned_addr learned my addr 172.21.15.114:0/2071387431 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45ff7fe700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46001587f0 0x7f460006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 -- 172.21.15.114:0/2071387431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46001587f0 msgr2=0x7f460006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46001587f0 0x7f460006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 -- 172.21.15.114:0/2071387431 >> v1:172.21.15.172:6789/0 conn(0x7f460015b370 legacy=0x7f46000b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 -- 172.21.15.114:0/2071387431 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460015c110 con 0x7f46000b4370 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fffff700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46000b4370 0x7f460015b7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009e50 tx=0x7f45f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a93c7a06e03ef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.120+0000 7f45fe7fc700 1 -- 172.21.15.114:0/2071387431 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800e070 con 0x7f46000b4370 2023-11-24T06:24:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f45fe7fc700 1 -- 172.21.15.114:0/2071387431 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800b9c0 con 0x7f46000b4370 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f45fe7fc700 1 -- 172.21.15.114:0/2071387431 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800bd70 con 0x7f46000b4370 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 -- 172.21.15.114:0/2071387431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46000b4370 msgr2=0x7f460015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46000b4370 0x7f460015b7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009e50 tx=0x7f45f800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 -- 172.21.15.114:0/2071387431 shutdown_connections 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46000b4370 0x7f460015b7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 --2- 172.21.15.114:0/2071387431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46001587f0 0x7f460006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 -- 172.21.15.114:0/2071387431 >> 172.21.15.114:0/2071387431 conn(0x7f46000d0bf0 msgr2=0x7f46000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:35.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 -- 172.21.15.114:0/2071387431 shutdown_connections 2023-11-24T06:24:35.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.121+0000 7f4605ef4700 1 -- 172.21.15.114:0/2071387431 wait complete. 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.123+0000 7f4605ef4700 1 Processor -- start 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.123+0000 7f4605ef4700 1 -- start start 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000b4370 0x7f460009aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46001587f0 0x7f46001424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600142280 con 0x7f460015b370 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600142a10 con 0x7f46001587f0 2023-11-24T06:24:35.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4600142cb0 con 0x7f46000b4370 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38386/0 (socket says 172.21.15.114:38386) 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 -- 172.21.15.114:0/3988773337 learned_addr learned my addr 172.21.15.114:0/3988773337 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45ff7fe700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000b4370 0x7f460009aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45feffd700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46001587f0 0x7f46001424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 -- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000b4370 msgr2=0x7f460009aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000b4370 0x7f460009aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 -- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46001587f0 msgr2=0x7f46001424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46001587f0 0x7f46001424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 -- 172.21.15.114:0/3988773337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8009b30 con 0x7f460015b370 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45fffff700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012000 tx=0x7f45f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004470 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8020070 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4600142f90 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004830 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.124+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4600143550 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.125+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46000ae1c0 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.126+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45f8022030 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.126+0000 7f45e7fff700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 0x7f45ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.126+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f808c880 con 0x7f460015b370 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.126+0000 7f45ff7fe700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 0x7f45ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.127+0000 7f45ff7fe700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 0x7f45ec06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f460006b500 tx=0x7f45f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.131+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45f8057940 con 0x7f460015b370 2023-11-24T06:24:35.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.324+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4600068fb0 con 0x7f460015b370 2023-11-24T06:24:35.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.324+0000 7f45e7fff700 1 -- 172.21.15.114:0/3988773337 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45f805af60 con 0x7f460015b370 2023-11-24T06:24:35.328 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 msgr2=0x7f45ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 0x7f45ec06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f460006b500 tx=0x7f45f4008040 comp rx=0 tx=0).stop 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 msgr2=0x7f4600143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45f8012000 tx=0x7f45f8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 shutdown_connections 2023-11-24T06:24:35.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45ec06aed0 0x7f45ec06d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460015b370 0x7f4600143d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000b4370 0x7f460009aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 --2- 172.21.15.114:0/3988773337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46001587f0 0x7f46001424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 >> 172.21.15.114:0/3988773337 conn(0x7f46000d0bf0 msgr2=0x7f4600071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.326+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 shutdown_connections 2023-11-24T06:24:35.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.327+0000 7f4605ef4700 1 -- 172.21.15.114:0/3988773337 wait complete. 2023-11-24T06:24:35.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:35.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.458+0000 7f261880c700 1 Processor -- start 2023-11-24T06:24:35.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.458+0000 7f261880c700 1 -- start start 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f261415bdd0 0x7f26141604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26140b3370 0x7f26140b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 0x7f26140d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140b2f50 con 0x7f261415bdd0 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140d2660 con 0x7f26140d1c70 2023-11-24T06:24:35.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261880c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140b2030 con 0x7f26140b3370 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261259c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f261415bdd0 0x7f26141604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261259c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f261415bdd0 0x7f26141604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38390/0 (socket says 172.21.15.114:38390) 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.459+0000 7f261259c700 1 -- 172.21.15.114:0/608930575 learned_addr learned my addr 172.21.15.114:0/608930575 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f2611d9b700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26140b3370 0x7f26140b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f2612d9d700 1 --2- 172.21.15.114:0/608930575 >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 0x7f26140d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 -- 172.21.15.114:0/608930575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26140b3370 msgr2=0x7f26140b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26140b3370 0x7f26140b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 -- 172.21.15.114:0/608930575 >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 msgr2=0x7f26140d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 --2- 172.21.15.114:0/608930575 >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 0x7f26140d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 -- 172.21.15.114:0/608930575 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140b22d0 con 0x7f261415bdd0 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f2612d9d700 1 --2- 172.21.15.114:0/608930575 >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 0x7f26140d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261259c700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f261415bdd0 0x7f26141604e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2614160a20 tx=0x7f2604009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d432c4ad54fc49 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261159a700 1 -- 172.21.15.114:0/608930575 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400b870 con 0x7f261415bdd0 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261159a700 1 -- 172.21.15.114:0/608930575 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400b9d0 con 0x7f261415bdd0 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.460+0000 7f261159a700 1 -- 172.21.15.114:0/608930575 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400bd60 con 0x7f261415bdd0 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 -- 172.21.15.114:0/608930575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f261415bdd0 msgr2=0x7f26141604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f261415bdd0 0x7f26141604e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2614160a20 tx=0x7f2604009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 -- 172.21.15.114:0/608930575 shutdown_connections 2023-11-24T06:24:35.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 --2- 172.21.15.114:0/608930575 >> v2:172.21.15.172:3300/0 conn(0x7f26140d1c70 0x7f26140d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f261415bdd0 0x7f26141604e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 --2- 172.21.15.114:0/608930575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26140b3370 0x7f26140b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 -- 172.21.15.114:0/608930575 >> 172.21.15.114:0/608930575 conn(0x7f26140731a0 msgr2=0x7f261409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 -- 172.21.15.114:0/608930575 shutdown_connections 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.461+0000 7f261880c700 1 -- 172.21.15.114:0/608930575 wait complete. 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.462+0000 7f261880c700 1 Processor -- start 2023-11-24T06:24:35.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.462+0000 7f261880c700 1 -- start start 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26140d1c70 0x7f2614156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f261415bdd0 0x7f261415c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140b6940 con 0x7f26140b3370 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614156800 con 0x7f26140d1c70 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140b5d30 con 0x7f261415bdd0 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38396/0 (socket says 172.21.15.114:38396) 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 -- 172.21.15.114:0/4258893893 learned_addr learned my addr 172.21.15.114:0/4258893893 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f2612d9d700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f261415bdd0 0x7f261415c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f2611d9b700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26140d1c70 0x7f2614156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 -- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f261415bdd0 msgr2=0x7f261415c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f261415bdd0 0x7f261415c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 -- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26140d1c70 msgr2=0x7f2614156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26140d1c70 0x7f2614156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 -- 172.21.15.114:0/4258893893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604009550 con 0x7f26140b3370 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261259c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26140716d0 tx=0x7f2604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604004540 con 0x7f26140b3370 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26040036a0 con 0x7f26140b3370 2023-11-24T06:24:35.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140b6010 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604003a50 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.463+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140b65d0 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.464+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26140b6c20 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.465+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2604021070 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.465+0000 7f2602ffd700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 0x7f25f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.465+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f260408ac90 con 0x7f26140b3370 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.465+0000 7f2611d9b700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 0x7f25f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.465+0000 7f2611d9b700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 0x7f25f406d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26140c5a20 tx=0x7f2608008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.469+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2604055dd0 con 0x7f26140b3370 2023-11-24T06:24:35.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.701+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2614067f60 con 0x7f26140b3370 2023-11-24T06:24:35.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.702+0000 7f2602ffd700 1 -- 172.21.15.114:0/4258893893 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f26040593f0 con 0x7f26140b3370 2023-11-24T06:24:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:35.706 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 msgr2=0x7f25f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 0x7f25f406d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26140c5a20 tx=0x7f2608008040 comp rx=0 tx=0).stop 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 msgr2=0x7f26141562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26140716d0 tx=0x7f2604004750 comp rx=0 tx=0).stop 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 shutdown_connections 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f25f406aed0 0x7f25f406d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f261415bdd0 0x7f261415c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26140b3370 0x7f26141562c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 --2- 172.21.15.114:0/4258893893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26140d1c70 0x7f2614156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 >> 172.21.15.114:0/4258893893 conn(0x7f26140731a0 msgr2=0x7f261415f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 shutdown_connections 2023-11-24T06:24:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.704+0000 7f261880c700 1 -- 172.21.15.114:0/4258893893 wait complete. 2023-11-24T06:24:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:24:35.720 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:35.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.836+0000 7f56f12b3700 1 Processor -- start 2023-11-24T06:24:35.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.836+0000 7f56f12b3700 1 -- start start 2023-11-24T06:24:35.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56f12b3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56ec15e1e0 0x7f56ec0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56f12b3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56ec0d0f80 0x7f56ec0b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56f12b3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0d20b0 con 0x7f56ec0d3a00 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56f12b3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0d3150 con 0x7f56ec15e1e0 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56f12b3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec15e5c0 con 0x7f56ec0d0f80 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56ec15e1e0 0x7f56ec0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56ec15e1e0 0x7f56ec0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38404/0 (socket says 172.21.15.114:38404) 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 -- 172.21.15.114:0/2810979131 learned_addr learned my addr 172.21.15.114:0/2810979131 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eb7fe700 1 --2- 172.21.15.114:0/2810979131 >> v2:172.21.15.172:3300/0 conn(0x7f56ec0d0f80 0x7f56ec0b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 -- 172.21.15.114:0/2810979131 >> v2:172.21.15.172:3300/0 conn(0x7f56ec0d0f80 msgr2=0x7f56ec0b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 --2- 172.21.15.114:0/2810979131 >> v2:172.21.15.172:3300/0 conn(0x7f56ec0d0f80 0x7f56ec0b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 -- 172.21.15.114:0/2810979131 >> v1:172.21.15.172:6789/0 conn(0x7f56ec0d3a00 legacy=0x7f56ec0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 -- 172.21.15.114:0/2810979131 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0d2390 con 0x7f56ec15e1e0 2023-11-24T06:24:35.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56eaffd700 1 --2- 172.21.15.114:0/2810979131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec0d34c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56ec08c640 tx=0x7f56dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c553e8937a481009 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56e9ffb700 1 -- 172.21.15.114:0/2810979131 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00b860 con 0x7f56ec15e1e0 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56e9ffb700 1 -- 172.21.15.114:0/2810979131 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc00b9c0 con 0x7f56ec15e1e0 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.837+0000 7f56e9ffb700 1 -- 172.21.15.114:0/2810979131 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00bd50 con 0x7f56ec15e1e0 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 -- 172.21.15.114:0/2810979131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 msgr2=0x7f56ec0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 --2- 172.21.15.114:0/2810979131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec0d34c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56ec08c640 tx=0x7f56dc009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 -- 172.21.15.114:0/2810979131 shutdown_connections 2023-11-24T06:24:35.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 --2- 172.21.15.114:0/2810979131 >> v2:172.21.15.172:3300/0 conn(0x7f56ec0d0f80 0x7f56ec0b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 --2- 172.21.15.114:0/2810979131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec0d34c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 -- 172.21.15.114:0/2810979131 >> 172.21.15.114:0/2810979131 conn(0x7f56ec0b5b50 msgr2=0x7f56ec0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 -- 172.21.15.114:0/2810979131 shutdown_connections 2023-11-24T06:24:35.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.838+0000 7f56f12b3700 1 -- 172.21.15.114:0/2810979131 wait complete. 2023-11-24T06:24:35.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.839+0000 7f56f12b3700 1 Processor -- start 2023-11-24T06:24:35.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.839+0000 7f56f12b3700 1 -- start start 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56ec0d0f80 0x7f56ec1f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec0d3a00 0x7f56ec157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0cbd30 con 0x7f56ec15e1e0 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0cb0b0 con 0x7f56ec0d3a00 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56f12b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0cb390 con 0x7f56ec0d0f80 2023-11-24T06:24:35.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38414/0 (socket says 172.21.15.114:38414) 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 -- 172.21.15.114:0/3497584461 learned_addr learned my addr 172.21.15.114:0/3497584461 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56ea7fc700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec0d3a00 0x7f56ec157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eaffd700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56ec0d0f80 0x7f56ec1f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 -- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56ec0d0f80 msgr2=0x7f56ec1f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56ec0d0f80 0x7f56ec1f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 -- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec0d3a00 msgr2=0x7f56ec157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec0d3a00 0x7f56ec157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.840+0000 7f56eb7fe700 1 -- 172.21.15.114:0/3497584461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009550 con 0x7f56ec15e1e0 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56eb7fe700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f56e4007e30 tx=0x7f56e400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e400eea0 con 0x7f56ec15e1e0 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e4014420 con 0x7f56ec15e1e0 2023-11-24T06:24:35.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec0cb670 con 0x7f56ec15e1e0 2023-11-24T06:24:35.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40147d0 con 0x7f56ec15e1e0 2023-11-24T06:24:35.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.841+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec15abc0 con 0x7f56ec15e1e0 2023-11-24T06:24:35.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.842+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec0751e0 con 0x7f56ec15e1e0 2023-11-24T06:24:35.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.842+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56e4014930 con 0x7f56ec15e1e0 2023-11-24T06:24:35.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.842+0000 7f56d37fe700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 0x7f56d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:35.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.843+0000 7f56eaffd700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 0x7f56d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:35.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.843+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56e4089d60 con 0x7f56ec15e1e0 2023-11-24T06:24:35.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.843+0000 7f56eaffd700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 0x7f56d806d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009c40 tx=0x7f56dc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:35.853 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:35.847+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56e4054ea0 con 0x7f56ec15e1e0 2023-11-24T06:24:36.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.042+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56ec0cbe70 con 0x7f56ec15e1e0 2023-11-24T06:24:36.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.044+0000 7f56d37fe700 1 -- 172.21.15.114:0/3497584461 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f56ec0cbe70 con 0x7f56ec15e1e0 2023-11-24T06:24:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:36.047 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:36.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 msgr2=0x7f56d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 0x7f56d806d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009c40 tx=0x7f56dc008030 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 msgr2=0x7f56ec15f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f56e4007e30 tx=0x7f56e400e3c0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 shutdown_connections 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f56d806aed0 0x7f56d806d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec15e1e0 0x7f56ec15f750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56ec0d0f80 0x7f56ec1f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 --2- 172.21.15.114:0/3497584461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec0d3a00 0x7f56ec157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 >> 172.21.15.114:0/3497584461 conn(0x7f56ec0b5b50 msgr2=0x7f56ec0b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 shutdown_connections 2023-11-24T06:24:36.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.046+0000 7f56f12b3700 1 -- 172.21.15.114:0/3497584461 wait complete. 2023-11-24T06:24:36.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:36.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.181+0000 7f77f988b700 1 Processor -- start 2023-11-24T06:24:36.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.181+0000 7f77f988b700 1 -- start start 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.182+0000 7f77f988b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77f41587f0 0x7f77f406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.182+0000 7f77f988b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 0x7f77f415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.182+0000 7f77f988b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f406b530 con 0x7f77f40b3c60 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.182+0000 7f77f988b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40ae5e0 con 0x7f77f41587f0 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.182+0000 7f77f988b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f415be30 con 0x7f77f415b370 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77f41587f0 0x7f77f406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f37fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f77f40b3c60 0x7f77f40af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47726/0 (socket says 172.21.15.114:47726) 2023-11-24T06:24:36.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f37fe700 1 -- 172.21.15.114:0/3280785597 learned_addr learned my addr 172.21.15.114:0/3280785597 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f1ffb700 1 -- 172.21.15.114:0/3280785597 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682120211 0 0) 0x7f77f406b530 con 0x7f77f40b3c60 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f1ffb700 1 -- 172.21.15.114:0/3280785597 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77dc0035b0 con 0x7f77f40b3c60 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f27fc700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 0x7f77f415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 -- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 msgr2=0x7f77f415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 0x7f77f415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 -- 172.21.15.114:0/3280785597 >> v1:172.21.15.172:6789/0 conn(0x7f77f40b3c60 legacy=0x7f77f40af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 -- 172.21.15.114:0/3280785597 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f415c110 con 0x7f77f41587f0 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f27fc700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 0x7f77f415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.183+0000 7f77f2ffd700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f41587f0 0x7f77f406aff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009840 tx=0x7f77e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c699a5d60849ce10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f1ffb700 1 -- 172.21.15.114:0/3280785597 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4011830 con 0x7f77f41587f0 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f1ffb700 1 -- 172.21.15.114:0/3280785597 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4011990 con 0x7f77f41587f0 2023-11-24T06:24:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f1ffb700 1 -- 172.21.15.114:0/3280785597 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4011d00 con 0x7f77f41587f0 2023-11-24T06:24:36.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 -- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f41587f0 msgr2=0x7f77f406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f41587f0 0x7f77f406aff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009840 tx=0x7f77e4009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:36.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 -- 172.21.15.114:0/3280785597 shutdown_connections 2023-11-24T06:24:36.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f41587f0 0x7f77f406aff0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 --2- 172.21.15.114:0/3280785597 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f415b370 0x7f77f415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 -- 172.21.15.114:0/3280785597 >> 172.21.15.114:0/3280785597 conn(0x7f77f40d0bf0 msgr2=0x7f77f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 -- 172.21.15.114:0/3280785597 shutdown_connections 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.184+0000 7f77f988b700 1 -- 172.21.15.114:0/3280785597 wait complete. 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.185+0000 7f77f988b700 1 Processor -- start 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 -- start start 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f40b3c60 0x7f77f4154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f415b370 0x7f77f4145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f414e9b0 con 0x7f77f41587f0 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f415c510 con 0x7f77f415b370 2023-11-24T06:24:36.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f988b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4153940 con 0x7f77f40b3c60 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38426/0 (socket says 172.21.15.114:38426) 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.186+0000 7f77f27fc700 1 -- 172.21.15.114:0/3259786152 learned_addr learned my addr 172.21.15.114:0/3259786152 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f2ffd700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f40b3c60 0x7f77f4154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 -- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f40b3c60 msgr2=0x7f77f4154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f40b3c60 0x7f77f4154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 -- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f415b370 msgr2=0x7f77f4145120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f415b370 0x7f77f4145120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 -- 172.21.15.114:0/3259786152 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40094a0 con 0x7f77f41587f0 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f27fc700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c7f0 tx=0x7f77e800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8011800 con 0x7f77f41587f0 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8011960 con 0x7f77f41587f0 2023-11-24T06:24:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4153b90 con 0x7f77f41587f0 2023-11-24T06:24:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8011cf0 con 0x7f77f41587f0 2023-11-24T06:24:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.187+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4154150 con 0x7f77f41587f0 2023-11-24T06:24:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.188+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f4154370 con 0x7f77f41587f0 2023-11-24T06:24:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.188+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77e8018360 con 0x7f77f41587f0 2023-11-24T06:24:36.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.189+0000 7f77dbfff700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 0x7f77e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.189+0000 7f77f2ffd700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 0x7f77e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.189+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e80894e0 con 0x7f77f41587f0 2023-11-24T06:24:36.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.189+0000 7f77f2ffd700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 0x7f77e006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.193+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77e80545a0 con 0x7f77f41587f0 2023-11-24T06:24:36.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.383+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77f4068fb0 con 0x7f77f41587f0 2023-11-24T06:24:36.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.383+0000 7f77dbfff700 1 -- 172.21.15.114:0/3259786152 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77e8057bc0 con 0x7f77f41587f0 2023-11-24T06:24:36.387 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:36.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 msgr2=0x7f77e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 0x7f77e006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e4008030 comp rx=0 tx=0).stop 2023-11-24T06:24:36.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 msgr2=0x7f77f4150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c7f0 tx=0x7f77e800cb00 comp rx=0 tx=0).stop 2023-11-24T06:24:36.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 shutdown_connections 2023-11-24T06:24:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77e006aed0 0x7f77e006d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77f415b370 0x7f77f4145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77f40b3c60 0x7f77f4154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 --2- 172.21.15.114:0/3259786152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77f41587f0 0x7f77f4150c80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.385+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 >> 172.21.15.114:0/3259786152 conn(0x7f77f40d0bf0 msgr2=0x7f77f40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.386+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 shutdown_connections 2023-11-24T06:24:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.386+0000 7f77f988b700 1 -- 172.21.15.114:0/3259786152 wait complete. 2023-11-24T06:24:36.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:36.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.521+0000 7fa25f608700 1 Processor -- start 2023-11-24T06:24:36.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.521+0000 7fa25f608700 1 -- start start 2023-11-24T06:24:36.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25f608700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa2580b4020 0x7fa258073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25f608700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580744d0 0x7fa25806d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25f608700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25815bf70 con 0x7fa2580af520 2023-11-24T06:24:36.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25f608700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa258074980 con 0x7fa2580b4020 2023-11-24T06:24:36.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25f608700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25815ee00 con 0x7fa2580744d0 2023-11-24T06:24:36.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa2580b4020 0x7fa258073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25dba5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580744d0 0x7fa25806d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa2580b4020 0x7fa258073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:34092/0 (socket says 172.21.15.114:34092) 2023-11-24T06:24:36.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25d3a4700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa2580af520 0x7fa2580afb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47736/0 (socket says 172.21.15.114:47736) 2023-11-24T06:24:36.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 -- 172.21.15.114:0/101767183 learned_addr learned my addr 172.21.15.114:0/101767183 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:36.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 -- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580744d0 msgr2=0x7fa25806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:36.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 --2- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580744d0 0x7fa25806d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 -- 172.21.15.114:0/101767183 >> v1:172.21.15.172:6789/0 conn(0x7fa2580af520 legacy=0x7fa2580afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa24ffff700 1 -- 172.21.15.114:0/101767183 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982994712 0 0) 0x7fa248001110 con 0x7fa2580af520 2023-11-24T06:24:36.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.522+0000 7fa25cba3700 1 -- 172.21.15.114:0/101767183 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580b0240 con 0x7fa2580b4020 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25cba3700 1 --2- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258073f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe738883ac44d94c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa24ffff700 1 -- 172.21.15.114:0/101767183 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011830 con 0x7fa2580b4020 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa24ffff700 1 -- 172.21.15.114:0/101767183 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254011990 con 0x7fa2580b4020 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa24ffff700 1 -- 172.21.15.114:0/101767183 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011d20 con 0x7fa2580b4020 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 -- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 msgr2=0x7fa258073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 --2- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258073f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 -- 172.21.15.114:0/101767183 shutdown_connections 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 --2- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580744d0 0x7fa25806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 --2- 172.21.15.114:0/101767183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258073f90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 -- 172.21.15.114:0/101767183 >> 172.21.15.114:0/101767183 conn(0x7fa2580b28f0 msgr2=0x7fa2580b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 -- 172.21.15.114:0/101767183 shutdown_connections 2023-11-24T06:24:36.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.523+0000 7fa25f608700 1 -- 172.21.15.114:0/101767183 wait complete. 2023-11-24T06:24:36.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 Processor -- start 2023-11-24T06:24:36.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 -- start start 2023-11-24T06:24:36.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580af520 0x7fa2581582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581f4a50 con 0x7fa2580744d0 2023-11-24T06:24:36.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258156d50 con 0x7fa2580b4020 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25f608700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2581f3ba0 con 0x7fa2580af520 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25d3a4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25d3a4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38438/0 (socket says 172.21.15.114:38438) 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25cba3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580af520 0x7fa2581582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25d3a4700 1 -- 172.21.15.114:0/1215398396 learned_addr learned my addr 172.21.15.114:0/1215398396 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.525+0000 7fa25dba5700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 -- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580af520 msgr2=0x7fa2581582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580af520 0x7fa2581582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 -- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 msgr2=0x7fa258158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 -- 172.21.15.114:0/1215398396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009550 con 0x7fa2580744d0 2023-11-24T06:24:36.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25d3a4700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c730 tx=0x7fa24800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248004c80 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248004e00 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2581f3e80 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248007310 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.526+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581f4440 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.527+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258068fb0 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.528+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa248007470 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.528+0000 7fa24dffb700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 0x7fa23c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.528+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa248089150 con 0x7fa2580744d0 2023-11-24T06:24:36.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.528+0000 7fa25cba3700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 0x7fa23c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.529+0000 7fa25cba3700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 0x7fa23c06d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.533+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa248054160 con 0x7fa2580744d0 2023-11-24T06:24:36.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.720+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa25815bf50 con 0x7fa2580744d0 2023-11-24T06:24:36.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.721+0000 7fa24dffb700 1 -- 172.21.15.114:0/1215398396 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa248057780 con 0x7fa2580744d0 2023-11-24T06:24:36.724 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 msgr2=0x7fa23c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 0x7fa23c06d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2540098f0 tx=0x7fa254008030 comp rx=0 tx=0).stop 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 msgr2=0x7fa2580b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa24800c730 tx=0x7fa24800ca40 comp rx=0 tx=0).stop 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 shutdown_connections 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa23c06aed0 0x7fa23c06d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2580b4020 0x7fa258158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2580744d0 0x7fa2580b1d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 --2- 172.21.15.114:0/1215398396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2580af520 0x7fa2581582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 >> 172.21.15.114:0/1215398396 conn(0x7fa2580b28f0 msgr2=0x7fa25815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 shutdown_connections 2023-11-24T06:24:36.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.723+0000 7fa25f608700 1 -- 172.21.15.114:0/1215398396 wait complete. 2023-11-24T06:24:36.739 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:36.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.860+0000 7f09ac878700 1 Processor -- start 2023-11-24T06:24:36.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.861+0000 7f09ac878700 1 -- start start 2023-11-24T06:24:36.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.861+0000 7f09ac878700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09a80746a0 0x7f09a80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.861+0000 7f09ac878700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80b3c60 con 0x7f09a80afc90 2023-11-24T06:24:36.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.861+0000 7f09ac878700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80b58c0 con 0x7f09a815e160 2023-11-24T06:24:36.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.861+0000 7f09ac878700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a80b4500 con 0x7f09a80746a0 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a6d9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f09a80afc90 0x7f09a806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54158/0 (socket says 172.21.15.114:54158) 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a6d9d700 1 -- 172.21.15.114:0/2783825823 learned_addr learned my addr 172.21.15.114:0/2783825823 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664998994 0 0) 0x7f09a80b3c60 con 0x7f09a80afc90 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a5d9b700 1 --2- 172.21.15.114:0/2783825823 >> v2:172.21.15.172:3300/0 conn(0x7f09a80746a0 0x7f09a80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09900035b0 con 0x7f09a80afc90 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866039714 0 0) 0x7f09900035b0 con 0x7f09a80afc90 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 >> v2:172.21.15.172:3300/0 conn(0x7f09a80746a0 msgr2=0x7f09a80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:36.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 --2- 172.21.15.114:0/2783825823 >> v2:172.21.15.172:3300/0 conn(0x7f09a80746a0 0x7f09a80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 >> v1:172.21.15.172:6789/0 conn(0x7f09a815e160 legacy=0x7f09a8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a806bdc0 con 0x7f09a80afc90 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f09a00019f0 con 0x7f09a80afc90 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 >> v1:172.21.15.114:6789/0 conn(0x7f09a80afc90 legacy=0x7f09a806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0990003a00 0x7f0990003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0990004850 0x7f0990004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f099000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80b4500 con 0x7f0990005210 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099000c0b0 con 0x7f0990003a00 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099000c270 con 0x7f0990004850 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a6d9d700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f099000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a659c700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0990003a00 0x7f0990003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.862+0000 7f09a5d9b700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0990004850 0x7f0990004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 -- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0990004850 msgr2=0x7f0990004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0990004850 0x7f0990004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 -- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0990003a00 msgr2=0x7f0990003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0990003a00 0x7f0990003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 -- 172.21.15.114:0/2783825823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900035b0 con 0x7f0990005210 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 -- 172.21.15.114:0/2783825823 reap_dead start 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a6d9d700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f099000ba10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09a000a820 tx=0x7f0990005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24539a81c83b967e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000a2f0 con 0x7f0990005210 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0012600 con 0x7f0990005210 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09a559a700 1 -- 172.21.15.114:0/2783825823 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00129b0 con 0x7f0990005210 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09ac878700 1 -- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 msgr2=0x7f099000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09ac878700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f099000ba10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09a000a820 tx=0x7f0990005880 comp rx=0 tx=0).stop 2023-11-24T06:24:36.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09ac878700 1 -- 172.21.15.114:0/2783825823 shutdown_connections 2023-11-24T06:24:36.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09ac878700 1 --2- 172.21.15.114:0/2783825823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f099000ba10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.863+0000 7f09ac878700 1 -- 172.21.15.114:0/2783825823 >> 172.21.15.114:0/2783825823 conn(0x7f09a80d0bf0 msgr2=0x7f09a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:36.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.864+0000 7f09ac878700 1 -- 172.21.15.114:0/2783825823 shutdown_connections 2023-11-24T06:24:36.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.864+0000 7f09ac878700 1 -- 172.21.15.114:0/2783825823 wait complete. 2023-11-24T06:24:36.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.865+0000 7f09ac878700 1 Processor -- start 2023-11-24T06:24:36.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.865+0000 7f09ac878700 1 -- start start 2023-11-24T06:24:36.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a815a260 0x7f09a814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a80b19a0 0x7f09a80b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80b2370 con 0x7f0990005210 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a815b270 con 0x7f09a815a260 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09ac878700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a815a6f0 con 0x7f09a80b19a0 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38458/0 (socket says 172.21.15.114:38458) 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 -- 172.21.15.114:0/787515150 learned_addr learned my addr 172.21.15.114:0/787515150 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a6d9d700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a80b19a0 0x7f09a80b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 -- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a80b19a0 msgr2=0x7f09a80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a5d9b700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a815a260 0x7f09a814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a80b19a0 0x7f09a80b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 -- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a815a260 msgr2=0x7f09a814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a815a260 0x7f09a814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.866+0000 7f09a659c700 1 -- 172.21.15.114:0/787515150 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a815a9d0 con 0x7f0990005210 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f09a659c700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f099c00f7f0 tx=0x7f099c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0052e0 con 0x7f0990005210 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c005440 con 0x7f0990005210 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a815acb0 con 0x7f0990005210 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0038a0 con 0x7f0990005210 2023-11-24T06:24:36.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.867+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a815c130 con 0x7f0990005210 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.868+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a80b0930 con 0x7f0990005210 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.868+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f099c003a00 con 0x7f0990005210 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.868+0000 7f0996ffd700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 0x7f098806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.868+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f099c089300 con 0x7f0990005210 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.868+0000 7f09a5d9b700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 0x7f098806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.869+0000 7f09a5d9b700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 0x7f098806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09a815b870 tx=0x7f0998008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:36.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:36.873+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f099c054440 con 0x7f0990005210 2023-11-24T06:24:37.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.068+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09a8068fb0 con 0x7f0990005210 2023-11-24T06:24:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.068+0000 7f0996ffd700 1 -- 172.21.15.114:0/787515150 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f099c003cb0 con 0x7f0990005210 2023-11-24T06:24:37.072 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 msgr2=0x7f098806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 0x7f098806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09a815b870 tx=0x7f0998008040 comp rx=0 tx=0).stop 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 msgr2=0x7f09a815b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f099c00f7f0 tx=0x7f099c00fbb0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 shutdown_connections 2023-11-24T06:24:37.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f098806aed0 0x7f098806d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09a80b19a0 0x7f09a80b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0990005210 0x7f09a815b920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 --2- 172.21.15.114:0/787515150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09a815a260 0x7f09a814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 >> 172.21.15.114:0/787515150 conn(0x7f09a80d0bf0 msgr2=0x7f09a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 shutdown_connections 2023-11-24T06:24:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.070+0000 7f09ac878700 1 -- 172.21.15.114:0/787515150 wait complete. 2023-11-24T06:24:37.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.202+0000 7fbfa27aa700 1 Processor -- start 2023-11-24T06:24:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.202+0000 7fbfa27aa700 1 -- start start 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa27aa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf9c06c710 0x7fbf9c165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa27aa700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0af950 con 0x7fbf9c1446a0 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa27aa700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0afc30 con 0x7fbf9c0adf00 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa27aa700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c144a80 con 0x7fbf9c06c710 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbf9bfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf9c1446a0 0x7fbf9c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54174/0 (socket says 172.21.15.114:54174) 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbf9bfff700 1 -- 172.21.15.114:0/2885468503 learned_addr learned my addr 172.21.15.114:0/2885468503 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa0d47700 1 --2- 172.21.15.114:0/2885468503 >> v2:172.21.15.172:3300/0 conn(0x7fbf9c06c710 0x7fbf9c165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbf9affd700 1 -- 172.21.15.114:0/2885468503 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706720372 0 0) 0x7fbf9c0af950 con 0x7fbf9c1446a0 2023-11-24T06:24:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbf9affd700 1 -- 172.21.15.114:0/2885468503 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf840035b0 con 0x7fbf9c1446a0 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa0d47700 1 -- 172.21.15.114:0/2885468503 >> v1:172.21.15.172:6789/0 conn(0x7fbf9c0adf00 legacy=0x7fbf9c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa0d47700 1 -- 172.21.15.114:0/2885468503 >> v1:172.21.15.114:6789/0 conn(0x7fbf9c1446a0 legacy=0x7fbf9c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.203+0000 7fbfa0d47700 1 -- 172.21.15.114:0/2885468503 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0aff10 con 0x7fbf9c06c710 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa0d47700 1 --2- 172.21.15.114:0/2885468503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c06c710 0x7fbf9c165b70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7dac02775a63a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbf9affd700 1 -- 172.21.15.114:0/2885468503 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400e070 con 0x7fbf9c06c710 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbf9affd700 1 -- 172.21.15.114:0/2885468503 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9400bb20 con 0x7fbf9c06c710 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbf9affd700 1 -- 172.21.15.114:0/2885468503 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94012430 con 0x7fbf9c06c710 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 -- 172.21.15.114:0/2885468503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c06c710 msgr2=0x7fbf9c165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/2885468503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c06c710 0x7fbf9c165b70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 -- 172.21.15.114:0/2885468503 shutdown_connections 2023-11-24T06:24:37.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/2885468503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c06c710 0x7fbf9c165b70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 -- 172.21.15.114:0/2885468503 >> 172.21.15.114:0/2885468503 conn(0x7fbf9c071b90 msgr2=0x7fbf9c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 -- 172.21.15.114:0/2885468503 shutdown_connections 2023-11-24T06:24:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.204+0000 7fbfa27aa700 1 -- 172.21.15.114:0/2885468503 wait complete. 2023-11-24T06:24:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.205+0000 7fbfa27aa700 1 Processor -- start 2023-11-24T06:24:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 -- start start 2023-11-24T06:24:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c0adf00 0x7fbf9c139600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf9c1446a0 0x7fbf9c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c139f70 con 0x7fbf9c06c710 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c13a250 con 0x7fbf9c0adf00 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbfa27aa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c13a530 con 0x7fbf9c1446a0 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.206+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38472/0 (socket says 172.21.15.114:38472) 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 -- 172.21.15.114:0/1724966339 learned_addr learned my addr 172.21.15.114:0/1724966339 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 -- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf9c1446a0 msgr2=0x7fbf9c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9b7fe700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c0adf00 0x7fbf9c139600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf9c1446a0 0x7fbf9c13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 -- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c0adf00 msgr2=0x7fbf9c139600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c0adf00 0x7fbf9c139600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 -- 172.21.15.114:0/1724966339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009a70 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf9bfff700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf90001eb0 tx=0x7fbf9000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90005d70 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf900070d0 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c144c60 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90007460 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.207+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c139b40 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.208+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c0753f0 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.209+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf900075c0 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.209+0000 7fbf98ff9700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 0x7fbf8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.209+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf90088b20 con 0x7fbf9c06c710 2023-11-24T06:24:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.209+0000 7fbf9b7fe700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 0x7fbf8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.209+0000 7fbf9b7fe700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 0x7fbf8806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c141bb0 tx=0x7fbf8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.213+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf90053bd0 con 0x7fbf9c06c710 2023-11-24T06:24:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.445+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf9c139cd0 con 0x7fbf9c06c710 2023-11-24T06:24:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.445+0000 7fbf98ff9700 1 -- 172.21.15.114:0/1724966339 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fbf90017090 con 0x7fbf9c06c710 2023-11-24T06:24:37.449 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:37.449 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:24:37.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 msgr2=0x7fbf8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 0x7fbf8806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c141bb0 tx=0x7fbf8c009040 comp rx=0 tx=0).stop 2023-11-24T06:24:37.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 msgr2=0x7fbf9c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf90001eb0 tx=0x7fbf9000c860 comp rx=0 tx=0).stop 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 shutdown_connections 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf8806aed0 0x7fbf8806d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf9c1446a0 0x7fbf9c13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9c06c710 0x7fbf9c15df30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 --2- 172.21.15.114:0/1724966339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf9c0adf00 0x7fbf9c139600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.447+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 >> 172.21.15.114:0/1724966339 conn(0x7fbf9c071b90 msgr2=0x7fbf9c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.448+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 shutdown_connections 2023-11-24T06:24:37.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.448+0000 7fbfa27aa700 1 -- 172.21.15.114:0/1724966339 wait complete. 2023-11-24T06:24:37.453 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:24:37.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.580+0000 7fcbac1be700 1 Processor -- start 2023-11-24T06:24:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.580+0000 7fcbac1be700 1 -- start start 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcbac1be700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 0x7fcba40622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcbac1be700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba40cfff0 con 0x7fcba4158a50 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcbac1be700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba40cfd10 con 0x7fcba40c9fc0 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcbac1be700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4161f50 con 0x7fcba4158670 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9759700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcba4158a50 0x7fcba40c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54188/0 (socket says 172.21.15.114:54188) 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9759700 1 -- 172.21.15.114:0/2577216336 learned_addr learned my addr 172.21.15.114:0/2577216336 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9f5a700 1 --2- 172.21.15.114:0/2577216336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 0x7fcba40622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711024751 0 0) 0x7fcba40cfff0 con 0x7fcba4158a50 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb900035b0 con 0x7fcba4158a50 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2684552132 0 0) 0x7fcb900035b0 con 0x7fcba4158a50 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9f5a700 1 -- 172.21.15.114:0/2577216336 >> v1:172.21.15.172:6789/0 conn(0x7fcba40c9fc0 legacy=0x7fcba40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9f5a700 1 -- 172.21.15.114:0/2577216336 >> v1:172.21.15.114:6789/0 conn(0x7fcba4158a50 legacy=0x7fcba40c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.581+0000 7fcba9f5a700 1 -- 172.21.15.114:0/2577216336 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4162230 con 0x7fcba4158670 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcba9f5a700 1 --2- 172.21.15.114:0/2577216336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 0x7fcba40622a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009870 tx=0x7fcba0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a9a4295e0bbada server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94001690 con 0x7fcba4158670 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0004030 con 0x7fcba4158670 2023-11-24T06:24:37.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcba8f58700 1 -- 172.21.15.114:0/2577216336 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00043a0 con 0x7fcba4158670 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 -- 172.21.15.114:0/2577216336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 msgr2=0x7fcba40622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 --2- 172.21.15.114:0/2577216336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 0x7fcba40622a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009870 tx=0x7fcba0009b80 comp rx=0 tx=0).stop 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 -- 172.21.15.114:0/2577216336 shutdown_connections 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 --2- 172.21.15.114:0/2577216336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba4158670 0x7fcba40622a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 -- 172.21.15.114:0/2577216336 >> 172.21.15.114:0/2577216336 conn(0x7fcba40b57b0 msgr2=0x7fcba40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 -- 172.21.15.114:0/2577216336 shutdown_connections 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.582+0000 7fcbac1be700 1 -- 172.21.15.114:0/2577216336 wait complete. 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.583+0000 7fcbac1be700 1 Processor -- start 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 -- start start 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba40c9fc0 0x7fcba415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcba4158a50 0x7fcba40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40cdb90 con 0x7fcba4158670 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b31b0 con 0x7fcba4158a50 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcbac1be700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b3490 con 0x7fcba40c9fc0 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcba9759700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcba9759700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38488/0 (socket says 172.21.15.114:38488) 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.584+0000 7fcba9759700 1 -- 172.21.15.114:0/802149128 learned_addr learned my addr 172.21.15.114:0/802149128 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9f5a700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba40c9fc0 0x7fcba415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcbaa75b700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcba4158a50 0x7fcba40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 -- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba40c9fc0 msgr2=0x7fcba415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba40c9fc0 0x7fcba415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 -- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcba4158a50 msgr2=0x7fcba40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcba4158a50 0x7fcba40cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 -- 172.21.15.114:0/802149128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00094a0 con 0x7fcba4158670 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcba9759700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb940013a0 tx=0x7fcb9400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004c80 con 0x7fcba4158670 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94004de0 con 0x7fcba4158670 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba40b3770 con 0x7fcba4158670 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007310 con 0x7fcba4158670 2023-11-24T06:24:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.585+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba415faf0 con 0x7fcba4158670 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.586+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba4067f60 con 0x7fcba4158670 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.586+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb94007470 con 0x7fcba4158670 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.587+0000 7fcb9a7fc700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 0x7fcb8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.587+0000 7fcba9f5a700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 0x7fcb8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.587+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb940892e0 con 0x7fcba4158670 2023-11-24T06:24:37.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.587+0000 7fcba9f5a700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 0x7fcb8806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcba00097c0 tx=0x7fcba0008000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.591+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb94054420 con 0x7fcba4158670 2023-11-24T06:24:37.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.784+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcba4074a20 con 0x7fcba4158670 2023-11-24T06:24:37.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.785+0000 7fcb9a7fc700 1 -- 172.21.15.114:0/802149128 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fcb94057a40 con 0x7fcba4158670 2023-11-24T06:24:37.789 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:37.789 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 msgr2=0x7fcb8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 0x7fcb8806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcba00097c0 tx=0x7fcba0008000 comp rx=0 tx=0).stop 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 msgr2=0x7fcba414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb940013a0 tx=0x7fcb9400ca90 comp rx=0 tx=0).stop 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 shutdown_connections 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb8806aed0 0x7fcb8806d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcba4158a50 0x7fcba40cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcba40c9fc0 0x7fcba415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 --2- 172.21.15.114:0/802149128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcba4158670 0x7fcba414b530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 >> 172.21.15.114:0/802149128 conn(0x7fcba40b57b0 msgr2=0x7fcba406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 shutdown_connections 2023-11-24T06:24:37.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.787+0000 7fcbac1be700 1 -- 172.21.15.114:0/802149128 wait complete. 2023-11-24T06:24:37.803 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:24:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.920+0000 7f6335f2c700 1 Processor -- start 2023-11-24T06:24:37.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.920+0000 7f6335f2c700 1 -- start start 2023-11-24T06:24:37.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f6335f2c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63301446a0 0x7f63300b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f6335f2c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63300adf00 0x7f633006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f6335f2c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63300af950 con 0x7f633006c540 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f6335f2c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6330144a80 con 0x7f63300adf00 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f6335f2c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633006c9b0 con 0x7f63301446a0 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f633006c540 0x7f633006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54204/0 (socket says 172.21.15.114:54204) 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632ffff700 1 -- 172.21.15.114:0/4206702894 learned_addr learned my addr 172.21.15.114:0/4206702894 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63301446a0 0x7f63300b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632effd700 1 --2- 172.21.15.114:0/4206702894 >> v2:172.21.15.172:3300/0 conn(0x7f63300adf00 0x7f633006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379038820 0 0) 0x7f63300af950 con 0x7f633006c540 2023-11-24T06:24:37.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f631c0035b0 con 0x7f633006c540 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839733944 0 0) 0x7f631c0035b0 con 0x7f633006c540 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63301446a0 msgr2=0x7f63300b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63301446a0 0x7f63300b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 >> v2:172.21.15.172:3300/0 conn(0x7f63300adf00 msgr2=0x7f633006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 --2- 172.21.15.114:0/4206702894 >> v2:172.21.15.172:3300/0 conn(0x7f63300adf00 0x7f633006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63300afc30 con 0x7f633006c540 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.921+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f63280019f0 con 0x7f633006c540 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 >> v1:172.21.15.114:6789/0 conn(0x7f633006c540 legacy=0x7f633006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f631c003a30 0x7f631c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c004850 0x7f631c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c005210 0x7f631c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330144a80 con 0x7f631c003a30 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633006c9b0 con 0x7f631c005210 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c00bf50 con 0x7f631c004850 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63301446a0 0x7f63300b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632effd700 1 --2- 172.21.15.114:0/4206702894 >> v2:172.21.15.172:3300/0 conn(0x7f63300adf00 0x7f633006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f631c003a30 0x7f631c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632ffff700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c005210 0x7f631c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632effd700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c004850 0x7f631c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 -- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c004850 msgr2=0x7f631c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c004850 0x7f631c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 -- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c005210 msgr2=0x7f631c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:37.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c005210 0x7f631c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 -- 172.21.15.114:0/4206702894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0035b0 con 0x7f631c003a30 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632f7fe700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f631c003a30 0x7f631c003e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6318009ba0 tx=0x7f631c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf689f0bdda3eeff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318004030 con 0x7f631c003a30 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318004190 con 0x7f631c003a30 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.922+0000 7f632e7fc700 1 -- 172.21.15.114:0/4206702894 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318004540 con 0x7f631c003a30 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 -- 172.21.15.114:0/4206702894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f631c003a30 msgr2=0x7f631c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 --2- 172.21.15.114:0/4206702894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f631c003a30 0x7f631c003e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6318009ba0 tx=0x7f631c005880 comp rx=0 tx=0).stop 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f632ffff700 1 -- 172.21.15.114:0/4206702894 reap_dead start 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 -- 172.21.15.114:0/4206702894 shutdown_connections 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 -- 172.21.15.114:0/4206702894 >> 172.21.15.114:0/4206702894 conn(0x7f6330071b90 msgr2=0x7f6330071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 -- 172.21.15.114:0/4206702894 shutdown_connections 2023-11-24T06:24:37.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.923+0000 7f6335f2c700 1 -- 172.21.15.114:0/4206702894 wait complete. 2023-11-24T06:24:37.932 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.924+0000 7f6335f2c700 1 Processor -- start 2023-11-24T06:24:37.932 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.924+0000 7f6335f2c700 1 -- start start 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c003a30 0x7f6330158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63300cf6d0 0x7f63300cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330159a70 con 0x7f63301590b0 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633009cf70 con 0x7f631c003a30 2023-11-24T06:24:37.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f6335f2c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633009c360 con 0x7f63300cf6d0 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38508/0 (socket says 172.21.15.114:38508) 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 -- 172.21.15.114:0/2765937823 learned_addr learned my addr 172.21.15.114:0/2765937823 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632f7fe700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c003a30 0x7f6330158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 -- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63300cf6d0 msgr2=0x7f63300cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632effd700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63300cf6d0 0x7f63300cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63300cf6d0 0x7f63300cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 -- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c003a30 msgr2=0x7f6330158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c003a30 0x7f6330158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 -- 172.21.15.114:0/2765937823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633009c640 con 0x7f63301590b0 2023-11-24T06:24:37.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.925+0000 7f632ffff700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f633015f8e0 tx=0x7f632800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.926+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800ad40 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.926+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280133f0 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.926+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280137a0 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.926+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633009c920 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.926+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633009c010 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.927+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63300753f0 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.927+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6328013900 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.927+0000 7f6327fff700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 0x7f631006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.928+0000 7f632f7fe700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 0x7f631006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.928+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6328089ee0 con 0x7f63301590b0 2023-11-24T06:24:37.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.928+0000 7f632f7fe700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 0x7f631006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f631800a010 tx=0x7f6318008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:37.937 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:37.932+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6328054f80 con 0x7f63301590b0 2023-11-24T06:24:38.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.167+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6330067f60 con 0x7f63301590b0 2023-11-24T06:24:38.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.167+0000 7f6327fff700 1 -- 172.21.15.114:0/2765937823 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f63280585a0 con 0x7f63301590b0 2023-11-24T06:24:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:38.172 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:24:38.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 msgr2=0x7f631006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 0x7f631006d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f631800a010 tx=0x7f6318008070 comp rx=0 tx=0).stop 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 msgr2=0x7f6330159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f633015f8e0 tx=0x7f632800a5f0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 shutdown_connections 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f631006aed0 0x7f631006d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63301590b0 0x7f6330159530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c003a30 0x7f6330158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 --2- 172.21.15.114:0/2765937823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63300cf6d0 0x7f63300cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 >> 172.21.15.114:0/2765937823 conn(0x7f6330071b90 msgr2=0x7f633006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 shutdown_connections 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.169+0000 7f6335f2c700 1 -- 172.21.15.114:0/2765937823 wait complete. 2023-11-24T06:24:38.175 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:24:38.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:24:19.243682+0000', 'modified': '2023-11-24T06:24:22.264928+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.114:6837/2364623566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6835', 'nonce': 2364623566}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 2364623566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:24:38.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:24:38.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:24:38.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.708+0000 7fbc97970700 1 Processor -- start 2023-11-24T06:24:38.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.708+0000 7fbc97970700 1 -- start start 2023-11-24T06:24:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc97970700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc9006c710 0x7fbc900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc97970700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 0x7fbc900afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc97970700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900ae430 con 0x7fbc9015e080 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc97970700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc900af0b0 con 0x7fbc9006c710 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc97970700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b05a0 con 0x7fbc900af4d0 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc94f0b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc9006c710 0x7fbc900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc94f0b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc9006c710 0x7fbc900b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38512/0 (socket says 172.21.15.114:38512) 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.709+0000 7fbc94f0b700 1 -- 172.21.15.114:0/3076192792 learned_addr learned my addr 172.21.15.114:0/3076192792 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 0x7fbc900afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc94f0b700 1 -- 172.21.15.114:0/3076192792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 msgr2=0x7fbc900afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 0x7fbc900afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc94f0b700 1 -- 172.21.15.114:0/3076192792 >> v1:172.21.15.172:6789/0 conn(0x7fbc9015e080 legacy=0x7fbc9006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc94f0b700 1 -- 172.21.15.114:0/3076192792 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900ae710 con 0x7fbc9006c710 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 0x7fbc900afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc900b5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098f0 tx=0x7fbc8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451774c8798ba8ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc87fff700 1 -- 172.21.15.114:0/3076192792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c004230 con 0x7fbc9006c710 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc87fff700 1 -- 172.21.15.114:0/3076192792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c004390 con 0x7fbc9006c710 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.710+0000 7fbc87fff700 1 -- 172.21.15.114:0/3076192792 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0036a0 con 0x7fbc9006c710 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 -- 172.21.15.114:0/3076192792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 msgr2=0x7fbc900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc900b5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098f0 tx=0x7fbc8c009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 -- 172.21.15.114:0/3076192792 shutdown_connections 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc900af4d0 0x7fbc900afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 --2- 172.21.15.114:0/3076192792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc900b5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 -- 172.21.15.114:0/3076192792 >> 172.21.15.114:0/3076192792 conn(0x7fbc90071b90 msgr2=0x7fbc90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 -- 172.21.15.114:0/3076192792 shutdown_connections 2023-11-24T06:24:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.711+0000 7fbc97970700 1 -- 172.21.15.114:0/3076192792 wait complete. 2023-11-24T06:24:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.712+0000 7fbc97970700 1 Processor -- start 2023-11-24T06:24:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.712+0000 7fbc97970700 1 -- start start 2023-11-24T06:24:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 0x7fbc900ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc9015e080 0x7fbc90156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015ed80 con 0x7fbc9006c710 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90153e60 con 0x7fbc900af4d0 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc97970700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90154140 con 0x7fbc9015e080 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38518/0 (socket says 172.21.15.114:38518) 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.713+0000 7fbc9570c700 1 -- 172.21.15.114:0/1430084454 learned_addr learned my addr 172.21.15.114:0/1430084454 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 -- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc9015e080 msgr2=0x7fbc90156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc9015e080 0x7fbc90156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 -- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 msgr2=0x7fbc900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 0x7fbc900ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 0x7fbc900ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 -- 172.21.15.114:0/1430084454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c009550 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 0x7fbc900ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b4a0 tx=0x7fbc8000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc85ffb700 1 -- 172.21.15.114:0/1430084454 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80011830 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc85ffb700 1 -- 172.21.15.114:0/1430084454 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc80011990 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9015e7c0 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc85ffb700 1 -- 172.21.15.114:0/1430084454 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80011d40 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.714+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc90154420 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.715+0000 7fbc85ffb700 1 -- 172.21.15.114:0/1430084454 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc8001a400 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.716+0000 7fbc85ffb700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 0x7fbc7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.716+0000 7fbc85ffb700 1 -- 172.21.15.114:0/1430084454 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc800892a0 con 0x7fbc9006c710 2023-11-24T06:24:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.716+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 0x7fbc7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.716+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 0x7fbc7406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc901482e0 tx=0x7fbc8c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.718+0000 7fbca3b69b80 1 Processor -- start 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.718+0000 7fbca3b69b80 1 -- start start 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b875ed0 0x56543b8a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b889e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b7d3770 con 0x56543b860080 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b7ba750 con 0x56543b875ed0 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543b7b8070 con 0x56543b85d490 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38526/0 (socket says 172.21.15.114:38526) 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 -- 172.21.15.114:0/1584058289 learned_addr learned my addr 172.21.15.114:0/1584058289 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 -- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 msgr2=0x56543b889e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b889e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 -- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b875ed0 msgr2=0x56543b8a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b875ed0 0x56543b8a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 -- 172.21.15.114:0/1584058289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543b6305e0 con 0x56543b860080 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.719+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc80087880 tx=0x7fbc80057a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc23f32267d2515c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbc7affd700 1 -- 172.21.15.114:0/1584058289 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800890c0 con 0x56543b860080 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbc7affd700 1 -- 172.21.15.114:0/1584058289 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8005c1f0 con 0x56543b860080 2023-11-24T06:24:38.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbc7affd700 1 -- 172.21.15.114:0/1584058289 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800603a0 con 0x56543b860080 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1584058289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 msgr2=0x56543b8a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc80087880 tx=0x7fbc80057a80 comp rx=0 tx=0).stop 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1584058289 shutdown_connections 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b889e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b860080 0x56543b8a6840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1584058289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b875ed0 0x56543b8a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1584058289 >> 172.21.15.114:0/1584058289 conn(0x56543b78e310 msgr2=0x56543b8a4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1584058289 shutdown_connections 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.720+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1584058289 wait complete. 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.721+0000 7fbca3b69b80 1 -- start start 2023-11-24T06:24:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.721+0000 7fbca3b69b80 1 Processor -- start 2023-11-24T06:24:38.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b85d490 0x56543b854330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 0x56543b854870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b7b3960 con 0x56543b875ed0 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b76d510 con 0x56543b85d490 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543b76d1b0 con 0x56543b860080 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbca3b69b80 10 client.0 _ll_register_callbacks cb 0x56543b698a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbc95f0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbc95f0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38536/0 (socket says 172.21.15.114:38536) 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbc95f0d700 1 -- 172.21.15.114:0/1125065654 learned_addr learned my addr 172.21.15.114:0/1125065654 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.722+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 0x56543b854870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 -- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 msgr2=0x56543b854870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b85d490 0x56543b854330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 0x56543b854870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 -- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b85d490 msgr2=0x56543b854330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b85d490 0x56543b854330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 -- 172.21.15.114:0/1125065654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc90154420 con 0x56543b875ed0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 0x56543b854870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800da70 tx=0x7fbc8800de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6508db8fbceb95dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800bc90 con 0x56543b875ed0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc8800bdf0 con 0x56543b875ed0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56543b852a40 con 0x56543b875ed0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc88016440 con 0x56543b875ed0 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 10 client.4701 fetch_fsmap learned FSMap version 8 2023-11-24T06:24:38.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56543b5dcd90 con 0x56543b875ed0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56543b852a40 con 0x56543b875ed0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc880165a0 con 0x56543b875ed0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 10 client.4701 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 10 client.4701 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 20 client.4701 populate_metadata read hostname 'smithi114' 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8a0ed0 0x56543b88c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 4 client.4701 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.723+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 --> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56543b82e840 con 0x56543b8a0ed0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8a0ed0 0x56543b88c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbc6ffff700 10 client.4701 ms_handle_connect on v2:172.21.15.114:6835/2364623566 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbc9570c700 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8a0ed0 0x56543b88c060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed540be31c645e31 server_cookie=f669449f1890cd7 in_seq=0 out_seq=0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbc6ffff700 1 -- 172.21.15.114:0/1125065654 <== mds.0 v2:172.21.15.114:6835/2364623566 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56543b82e840 con 0x56543b8a0ed0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbc6ffff700 10 client.4701 handle_command_reply: tid=0 2023-11-24T06:24:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbca3b69b80 1 client.4701 shutdown 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 msgr2=0x56543b8a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.724+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800da70 tx=0x7fbc8800de30 comp rx=0 tx=0).stop 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 shutdown_connections 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8a0ed0 0x56543b88c060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a04a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbc6ffff700 0 client.4701 ms_handle_reset on v2:172.21.15.114:6835/2364623566 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b85d490 0x56543b854330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/1125065654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b860080 0x56543b854870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 >> 172.21.15.114:0/1125065654 conn(0x56543b78e310 msgr2=0x56543b8a8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 shutdown_connections 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 1 -- 172.21.15.114:0/1125065654 wait complete. 2023-11-24T06:24:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.725+0000 7fbca3b69b80 20 client.4701 trim_cache size 0 max 16384 2023-11-24T06:24:38.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.742+0000 7fbca3b69b80 1 Processor -- start 2023-11-24T06:24:38.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 -- start start 2023-11-24T06:24:38.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b85cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b85a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b6328d0 con 0x56543b875ed0 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b85aea0 con 0x56543b860080 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543b860560 con 0x56543b85d490 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbc95f0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbc95f0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38544/0 (socket says 172.21.15.114:38544) 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbc95f0d700 1 -- 172.21.15.114:0/3537240453 learned_addr learned my addr 172.21.15.114:0/3537240453 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b85a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.743+0000 7fbc9570c700 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b85cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 -- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 msgr2=0x56543b85cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b85cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 -- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 msgr2=0x56543b85a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b85a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 -- 172.21.15.114:0/3537240453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543b5c5c10 con 0x56543b875ed0 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800de70 tx=0x7fbc88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed71372c11f93c69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc7b7fe700 1 -- 172.21.15.114:0/3537240453 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800b630 con 0x56543b875ed0 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc7b7fe700 1 -- 172.21.15.114:0/3537240453 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88016a90 con 0x56543b875ed0 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbc7b7fe700 1 -- 172.21.15.114:0/3537240453 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88016640 con 0x56543b875ed0 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 -- 172.21.15.114:0/3537240453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 msgr2=0x56543b8a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800de70 tx=0x7fbc88010000 comp rx=0 tx=0).stop 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 -- 172.21.15.114:0/3537240453 shutdown_connections 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b875ed0 0x56543b8a14a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b85d490 0x56543b85cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/3537240453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b85a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 -- 172.21.15.114:0/3537240453 >> 172.21.15.114:0/3537240453 conn(0x56543b78e310 msgr2=0x56543b85bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 -- 172.21.15.114:0/3537240453 shutdown_connections 2023-11-24T06:24:38.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.744+0000 7fbca3b69b80 1 -- 172.21.15.114:0/3537240453 wait complete. 2023-11-24T06:24:38.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.745+0000 7fbca3b69b80 1 -- start start 2023-11-24T06:24:38.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.745+0000 7fbca3b69b80 1 Processor -- start 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b8b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b875ed0 0x56543b89a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b8bde30 con 0x56543b85d490 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543b8c7620 con 0x56543b860080 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543b8c7130 con 0x56543b875ed0 2023-11-24T06:24:38.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 10 client.0 _ll_register_callbacks cb 0x56543b698a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38546/0 (socket says 172.21.15.114:38546) 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 -- 172.21.15.114:0/4039702434 learned_addr learned my addr 172.21.15.114:0/4039702434 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b8b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc95f0d700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b875ed0 0x56543b89a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 -- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b875ed0 msgr2=0x56543b89a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b875ed0 0x56543b89a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 -- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 msgr2=0x56543b8b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b8b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 -- 172.21.15.114:0/4039702434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56543b5dcd90 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc9570c700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc80058030 tx=0x7fbc80089050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53762d09995f86e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800890c0 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc800603a0 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.746+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc8005c1f0 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc80056440 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 10 client.4707 fetch_fsmap learned FSMap version 8 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56543b603200 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8005c1f0 con 0x56543b85d490 2023-11-24T06:24:38.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc8005e0e0 con 0x56543b85d490 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 10 client.4707 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 10 client.4707 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 20 client.4707 populate_metadata read hostname 'smithi114' 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8b3790 0x56543b89ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 4 client.4707 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 --> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56543b779f80 con 0x56543b8b3790 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8b3790 0x56543b89ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 10 client.4707 ms_handle_connect on v2:172.21.15.114:6835/2364623566 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc94f0b700 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8b3790 0x56543b89ad40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a783ec9af0def9 server_cookie=b67a49ccd32004a0 in_seq=0 out_seq=0 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 1 -- 172.21.15.114:0/4039702434 <== mds.0 v2:172.21.15.114:6835/2364623566 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56543b779f80 con 0x56543b8b3790 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.747+0000 7fbc6ffff700 10 client.4707 handle_command_reply: tid=0 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 client.4707 shutdown 2023-11-24T06:24:38.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 msgr2=0x56543b8a8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc80058030 tx=0x7fbc80089050 comp rx=0 tx=0).stop 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 shutdown_connections 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:6835/2364623566,v1:172.21.15.114:6837/2364623566] conn(0x56543b8b3790 0x56543b89ad40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56543b875ed0 0x56543b89a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc6ffff700 0 client.4707 ms_handle_reset on v2:172.21.15.114:6835/2364623566 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56543b85d490 0x56543b8a8b10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 --2- 172.21.15.114:0/4039702434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56543b860080 0x56543b8b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 >> 172.21.15.114:0/4039702434 conn(0x56543b78e310 msgr2=0x56543b8a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 shutdown_connections 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 1 -- 172.21.15.114:0/4039702434 wait complete. 2023-11-24T06:24:38.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbca3b69b80 20 client.4707 trim_cache size 0 max 16384 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "cfb0acb2-02f2-43f9-bba4-c35657dcf6fd", 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 4263, 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 18.883007069000001, 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 8, 2023-11-24T06:24:38.759 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 21, 2023-11-24T06:24:38.760 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 21, 2023-11-24T06:24:38.760 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 28.480332177000001 2023-11-24T06:24:38.760 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:24:38.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 msgr2=0x7fbc7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 0x7fbc7406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc901482e0 tx=0x7fbc8c008030 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 msgr2=0x7fbc90141b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b4a0 tx=0x7fbc8000b860 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 shutdown_connections 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc7406aed0 0x7fbc7406d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc9015e080 0x7fbc90156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc9006c710 0x7fbc90141b70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 --2- 172.21.15.114:0/1430084454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc900af4d0 0x7fbc900ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 >> 172.21.15.114:0/1430084454 conn(0x7fbc90071b90 msgr2=0x7fbc9006be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.748+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 shutdown_connections 2023-11-24T06:24:38.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:38.749+0000 7fbc97970700 1 -- 172.21.15.114:0/1430084454 wait complete. 2023-11-24T06:24:38.766 INFO:teuthology.run_tasks:Running task kclient... 2023-11-24T06:24:38.794 INFO:tasks.kclient:Mounting kernel clients... 2023-11-24T06:24:38.794 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:24:38.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:38.794 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:24:38.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:38.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2023-11-24T06:24:38.880 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2023-11-24T06:24:38.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:24:38.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:38.881 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:24:38.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:38.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2023-11-24T06:24:39.007 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2023-11-24T06:24:39.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:24:39.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.008 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:24:39.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-11-24T06:24:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-11-24T06:24:39.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:24:39.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.106 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:24:39.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-11-24T06:24:39.238 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-11-24T06:24:39.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:24:39.239 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-24T06:24:39.239 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:24:39.239 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:24:39.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:24:39.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.357+0000 7fb51ce5c700 1 Processor -- start 2023-11-24T06:24:39.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.357+0000 7fb51ce5c700 1 -- start start 2023-11-24T06:24:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51ce5c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb5180d2160 0x7fb5180b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51ce5c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180cd100 con 0x7fb51815bed0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51ce5c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180d09a0 con 0x7fb5181603c0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51ce5c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51815c2b0 con 0x7fb5180d2160 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51659c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb51815bed0 0x7fb5180d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54262/0 (socket says 172.21.15.114:54262) 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.358+0000 7fb51659c700 1 -- 172.21.15.114:0/4233960312 learned_addr learned my addr 172.21.15.114:0/4233960312 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb515d9b700 1 --2- 172.21.15.114:0/4233960312 >> v2:172.21.15.172:3300/0 conn(0x7fb5180d2160 0x7fb5180b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156929145 0 0) 0x7fb5180cd100 con 0x7fb51815bed0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5000035b0 con 0x7fb51815bed0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940551054 0 0) 0x7fb5000035b0 con 0x7fb51815bed0 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 >> v2:172.21.15.172:3300/0 conn(0x7fb5180d2160 msgr2=0x7fb5180b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 --2- 172.21.15.114:0/4233960312 >> v2:172.21.15.172:3300/0 conn(0x7fb5180d2160 0x7fb5180b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 >> v1:172.21.15.172:6789/0 conn(0x7fb5181603c0 legacy=0x7fb5180cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518160850 con 0x7fb51815bed0 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fb508001670 con 0x7fb51815bed0 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 >> v1:172.21.15.114:6789/0 conn(0x7fb51815bed0 legacy=0x7fb5180d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 0x7fb500003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500004850 0x7fb500004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb500005210 0x7fb50000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815c2b0 con 0x7fb500003a00 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000c0b0 con 0x7fb500004850 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000c270 con 0x7fb500005210 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 0x7fb500003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb515d9b700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500004850 0x7fb500004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb516d9d700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb500005210 0x7fb50000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 -- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb500005210 msgr2=0x7fb50000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb500005210 0x7fb50000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 -- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500004850 msgr2=0x7fb500004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500004850 0x7fb500004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 -- 172.21.15.114:0/4233960312 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000035b0 con 0x7fb500003a00 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb516d9d700 1 -- 172.21.15.114:0/4233960312 reap_dead start 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.359+0000 7fb51659c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 0x7fb500003e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c240 tx=0x7fb500005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d97d701429609 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508007c10 con 0x7fb500003a00 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800ce00 con 0x7fb500003a00 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51559a700 1 -- 172.21.15.114:0/4233960312 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508005ba0 con 0x7fb500003a00 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 -- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 msgr2=0x7fb500003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 0x7fb500003e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c240 tx=0x7fb500005880 comp rx=0 tx=0).stop 2023-11-24T06:24:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 -- 172.21.15.114:0/4233960312 shutdown_connections 2023-11-24T06:24:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/4233960312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb500003a00 0x7fb500003e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 -- 172.21.15.114:0/4233960312 >> 172.21.15.114:0/4233960312 conn(0x7fb518073130 msgr2=0x7fb51809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 -- 172.21.15.114:0/4233960312 shutdown_connections 2023-11-24T06:24:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.360+0000 7fb51ce5c700 1 -- 172.21.15.114:0/4233960312 wait complete. 2023-11-24T06:24:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 Processor -- start 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 -- start start 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500003a00 0x7fb5180cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 0x7fb5181580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518150710 con 0x7fb51806cca0 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51806c9a0 con 0x7fb500003a00 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb51ce5c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180c58b0 con 0x7fb5180c6950 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb515d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb515d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38568/0 (socket says 172.21.15.114:38568) 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.362+0000 7fb515d9b700 1 -- 172.21.15.114:0/1427863666 learned_addr learned my addr 172.21.15.114:0/1427863666 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb51659c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500003a00 0x7fb5180cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb516d9d700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 0x7fb5181580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 -- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 msgr2=0x7fb5181580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 0x7fb5181580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 -- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500003a00 msgr2=0x7fb5180cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500003a00 0x7fb5180cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 -- 172.21.15.114:0/1427863666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180c5b90 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb516d9d700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 0x7fb5181580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb515d9b700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb50c011a20 tx=0x7fb50c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00f940 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00faa0 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5180c5e70 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00fe30 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.363+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5180c6430 con 0x7fb51806cca0 2023-11-24T06:24:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.364+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb518067f60 con 0x7fb51806cca0 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.364+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb50c014440 con 0x7fb51806cca0 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.365+0000 7fb506ffd700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 0x7fb4f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.365+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb50c089c80 con 0x7fb51806cca0 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.365+0000 7fb51659c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 0x7fb4f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.365+0000 7fb51659c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 0x7fb4f806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb508000c00 tx=0x7fb508007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.369+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb50c054dc0 con 0x7fb51806cca0 2023-11-24T06:24:39.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.605+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5180b3240 con 0x7fb51806cca0 2023-11-24T06:24:39.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.605+0000 7fb506ffd700 1 -- 172.21.15.114:0/1427863666 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb50c0583e0 con 0x7fb51806cca0 2023-11-24T06:24:39.609 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:24:39.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.607+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 msgr2=0x7fb4f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.607+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 0x7fb4f806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb508000c00 tx=0x7fb508007000 comp rx=0 tx=0).stop 2023-11-24T06:24:39.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.607+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 msgr2=0x7fb5181501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.607+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb50c011a20 tx=0x7fb50c011d30 comp rx=0 tx=0).stop 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 shutdown_connections 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb4f806aed0 0x7fb4f806d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5180c6950 0x7fb5181580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb500003a00 0x7fb5180cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 --2- 172.21.15.114:0/1427863666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb51806cca0 0x7fb5181501d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 >> 172.21.15.114:0/1427863666 conn(0x7fb518073130 msgr2=0x7fb5180ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 shutdown_connections 2023-11-24T06:24:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:39.608+0000 7fb51ce5c700 1 -- 172.21.15.114:0/1427863666 wait complete. 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:24:39.623 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:24:39.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.624 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:24:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:24:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-11-24T06:24:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::9ca9:e760:b5c4:9eac/64 scope link noprefixroute 2023-11-24T06:24:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:39.643 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:24:39.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2023-11-24T06:24:39.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:24:39.645 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:39.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.809 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:24:39.836 INFO:teuthology.orchestra.run.smithi114.stdout:1 2023-11-24T06:24:39.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.838 DEBUG:teuthology.orchestra.run.smithi114:> route 2023-11-24T06:24:39.914 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2023-11-24T06:24:39.914 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:24:39.915 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:24:39.915 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:24:39.915 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:24:39.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:39.915 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:39.915 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:39.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:24:39.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:24:39.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:24:39.916 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:40.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.164 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:24:40.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.182 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2023-11-24T06:24:40.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.238 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:40.238 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:40.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:24:40.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:24:40.238 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:40.332 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:24:40.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:24:40.333 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:40.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.531 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:40.531 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:40.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2023-11-24T06:24:40.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:24:40.531 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:40.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:40.638 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:24:40.659 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:24:40.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:40.659 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:24:40.716 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:24:40.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:40.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:40.808 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:40.808 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:40.925 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:40.925 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:40.925 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:41.010 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:41.010 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:24:41.011 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:24:41.013 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:24:41.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:24:41.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.041 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:41.122 INFO:teuthology.orchestra.run.smithi114.stdout:client4487 2023-11-24T06:24:41.122 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:41.123 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:41.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.180 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:41.260 INFO:teuthology.orchestra.run.smithi114.stdout:client4487 2023-11-24T06:24:41.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.261 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:41.342 INFO:teuthology.orchestra.run.smithi114.stdout:client4487 2023-11-24T06:24:41.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.342 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.client4487/status') 2023-11-24T06:24:41.434 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.4487 (3)172.21.15.114:0/1647273205 2023-11-24T06:24:41.434 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2023-11-24T06:24:41.435 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2023-11-24T06:24:41.435 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2023-11-24T06:24:41.435 INFO:teuthology.orchestra.run.smithi114.stderr:71 bytes copied, 3.8018e-05 s, 1.9 MB/s 2023-11-24T06:24:41.435 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-24T06:24:41.435 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:24:41.435 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:24:41.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:24:41.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.552+0000 7fd9c4543700 1 Processor -- start 2023-11-24T06:24:41.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.552+0000 7fd9c4543700 1 -- start start 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c4543700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd9bc0b0ba0 0x7fd9bc0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c4543700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc15bb60 con 0x7fd9bc15ee90 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c4543700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc0b2bd0 con 0x7fd9bc0b0f80 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c4543700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc0b1fc0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c22df700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd9bc0b0ba0 0x7fd9bc0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c2ae0700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd9bc15ee90 0x7fd9bc15b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54286/0 (socket says 172.21.15.114:54286) 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c2ae0700 1 -- 172.21.15.114:0/3662067975 learned_addr learned my addr 172.21.15.114:0/3662067975 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c22df700 1 -- 172.21.15.114:0/3662067975 >> v1:172.21.15.172:6789/0 conn(0x7fd9bc0b0f80 legacy=0x7fd9bc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c22df700 1 -- 172.21.15.114:0/3662067975 >> v1:172.21.15.114:6789/0 conn(0x7fd9bc15ee90 legacy=0x7fd9bc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c12dd700 1 -- 172.21.15.114:0/3662067975 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125074153 0 0) 0x7fd9b4001490 con 0x7fd9bc15ee90 2023-11-24T06:24:41.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c22df700 1 -- 172.21.15.114:0/3662067975 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0b22a0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c22df700 1 --2- 172.21.15.114:0/3662067975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc0af5c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009840 tx=0x7fd9b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430afa947f67b7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:41.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c12dd700 1 -- 172.21.15.114:0/3662067975 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8011830 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c12dd700 1 -- 172.21.15.114:0/3662067975 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8011990 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.553+0000 7fd9c12dd700 1 -- 172.21.15.114:0/3662067975 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8011d20 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 -- 172.21.15.114:0/3662067975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 msgr2=0x7fd9bc0af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3662067975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc0af5c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009840 tx=0x7fd9b8009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:41.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 -- 172.21.15.114:0/3662067975 shutdown_connections 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3662067975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc0af5c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 -- 172.21.15.114:0/3662067975 >> 172.21.15.114:0/3662067975 conn(0x7fd9bc0b57b0 msgr2=0x7fd9bc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 -- 172.21.15.114:0/3662067975 shutdown_connections 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.554+0000 7fd9c4543700 1 -- 172.21.15.114:0/3662067975 wait complete. 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.555+0000 7fd9c4543700 1 Processor -- start 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.555+0000 7fd9c4543700 1 -- start start 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc0b0f80 0x7fd9bc099950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc15ee90 0x7fd9bc09c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc09c2c0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc099e90 con 0x7fd9bc0b0f80 2023-11-24T06:24:41.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c4543700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc09b640 con 0x7fd9bc15ee90 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38582/0 (socket says 172.21.15.114:38582) 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 -- 172.21.15.114:0/3995931553 learned_addr learned my addr 172.21.15.114:0/3995931553 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c1ade700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc0b0f80 0x7fd9bc099950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c2ae0700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc15ee90 0x7fd9bc09c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 -- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc15ee90 msgr2=0x7fd9bc09c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc15ee90 0x7fd9bc09c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 -- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc0b0f80 msgr2=0x7fd9bc099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc0b0f80 0x7fd9bc099950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.556+0000 7fd9c22df700 1 -- 172.21.15.114:0/3995931553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80094a0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9c22df700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80090b0 tx=0x7fd9b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b801a850 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b801a9b0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc09b920 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b801ad20 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.557+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc09bee0 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.558+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc0b1d70 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.558+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9b8022070 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.558+0000 7fd9b2ffd700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 0x7fd9a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.559+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9b808a730 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.559+0000 7fd9c1ade700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 0x7fd9a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:41.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.559+0000 7fd9c1ade700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 0x7fd9a006d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc15b0f0 tx=0x7fd9ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:41.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.563+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9b8055920 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.799+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9bc067f60 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.799+0000 7fd9b2ffd700 1 -- 172.21.15.114:0/3995931553 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd9b8058f40 con 0x7fd9bc0b0ba0 2023-11-24T06:24:41.803 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 msgr2=0x7fd9a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 0x7fd9a006d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc15b0f0 tx=0x7fd9ac009040 comp rx=0 tx=0).stop 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 msgr2=0x7fd9bc06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80090b0 tx=0x7fd9b8004750 comp rx=0 tx=0).stop 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 shutdown_connections 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd9a006aed0 0x7fd9a006d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc15ee90 0x7fd9bc09c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9bc0b0ba0 0x7fd9bc06ee90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 --2- 172.21.15.114:0/3995931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc0b0f80 0x7fd9bc099950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:41.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 >> 172.21.15.114:0/3995931553 conn(0x7fd9bc0b57b0 msgr2=0x7fd9bc15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:41.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.801+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 shutdown_connections 2023-11-24T06:24:41.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:41.802+0000 7fd9c4543700 1 -- 172.21.15.114:0/3995931553 wait complete. 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:24:41.817 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:24:41.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.818 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2023-11-24T06:24:41.834 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:41.834 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:41.834 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:24:41.835 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42373sec preferred_lft 42373sec 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::9ca9:e760:b5c4:9eac/64 scope link noprefixroute 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether fa:2a:57:62:51:8a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::385c:94ff:fec5:c8f4/64 scope link 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether fa:2a:57:62:51:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::f82a:57ff:fe62:518a/64 scope link tentative 2023-11-24T06:24:41.836 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:41.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.837 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:24:41.899 INFO:teuthology.orchestra.run.smithi114.stdout:1 2023-11-24T06:24:41.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.901 DEBUG:teuthology.orchestra.run.smithi114:> route 2023-11-24T06:24:41.960 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2023-11-24T06:24:41.960 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:24:41.960 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:24:41.961 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:24:41.961 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:24:41.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:24:41.961 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:42.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.066 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:24:42.084 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:24:42.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.084 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2023-11-24T06:24:42.140 INFO:teuthology.orchestra.run.smithi114.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:24:42.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.141 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:42.141 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:42.141 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:24:42.141 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:24:42.141 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.235 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:42.271 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether c2:ba:a3:53:6e:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::c0ba:a3ff:fe53:6ed5/64 scope link 2023-11-24T06:24:42.272 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:42.358 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether c2:ba:a3:53:6e:d5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::c0ba:a3ff:fe53:6ed5/64 scope link 2023-11-24T06:24:42.359 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:42.367 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:24:42.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:24:42.368 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.630 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:24:42.630 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:24:42.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.1 up 2023-11-24T06:24:42.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:24:42.630 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:24:42.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:42.740 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:24:42.765 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:24:42.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:42.765 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:24:42.821 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-24T06:24:42.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.822 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:42.907 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:42.907 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:42.930 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 2@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:42.931 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:42.962 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:42.962 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:42.962 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:42.962 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 2@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:24:42.963 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:24:42.970 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:24:42.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:24:42.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:42.998 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:43.070 INFO:teuthology.orchestra.run.smithi114.stdout:client4725 2023-11-24T06:24:43.070 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:43.070 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:43.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:43.128 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:43.200 INFO:teuthology.orchestra.run.smithi114.stdout:client4725 2023-11-24T06:24:43.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:43.201 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:43.275 INFO:teuthology.orchestra.run.smithi114.stdout:client4725 2023-11-24T06:24:43.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:43.276 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.client4725/status') 2023-11-24T06:24:43.355 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.4725 (3)192.168.0.2:0/3431498165 2023-11-24T06:24:43.355 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2023-11-24T06:24:43.356 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2023-11-24T06:24:43.356 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2023-11-24T06:24:43.356 INFO:teuthology.orchestra.run.smithi114.stderr:69 bytes copied, 1.9863e-05 s, 3.5 MB/s 2023-11-24T06:24:43.357 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-24T06:24:43.357 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:24:43.357 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:24:43.357 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:24:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.513+0000 7fbab722f700 1 Processor -- start 2023-11-24T06:24:43.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.513+0000 7fbab722f700 1 -- start start 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab722f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbab00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab722f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00ae430 con 0x7fbab015e080 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab722f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00ae710 con 0x7fbab006c710 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab722f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00af0b0 con 0x7fbab00af4d0 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab4fcb700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbab015e080 0x7fbab006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54304/0 (socket says 172.21.15.114:54304) 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab4fcb700 1 -- 172.21.15.114:0/1949481721 learned_addr learned my addr 172.21.15.114:0/1949481721 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782371469 0 0) 0x7fbab00ae430 con 0x7fbab015e080 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbab00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.514+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa00035b0 con 0x7fbab015e080 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675162409 0 0) 0x7fbaa00035b0 con 0x7fbab015e080 2023-11-24T06:24:43.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 msgr2=0x7fbab00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbab00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 >> v1:172.21.15.172:6789/0 conn(0x7fbab006c710 legacy=0x7fbab006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00ae9f0 con 0x7fbab015e080 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fbaa4001670 con 0x7fbab015e080 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 >> v1:172.21.15.114:6789/0 conn(0x7fbab015e080 legacy=0x7fbab006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbaa0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaa0003fb0 0x7fbaa0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaa0004d90 0x7fbaa0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00af0b0 con 0x7fbaa0004d90 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa000baf0 con 0x7fbaa0003fb0 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa000bcb0 con 0x7fbab00af4d0 2023-11-24T06:24:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaa0004d90 0x7fbaa0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab4fcb700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbaa0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaffff700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaa0003fb0 0x7fbaa0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 -- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 msgr2=0x7fbaa0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab00af4d0 0x7fbaa0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 -- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaa0003fb0 msgr2=0x7fbaa0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaa0003fb0 0x7fbaa0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 -- 172.21.15.114:0/1949481721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00035b0 con 0x7fbaa0004d90 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaa0004d90 0x7fbaa0005210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009fc0 tx=0x7fbaa0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14959cfc82d2fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800ec30 con 0x7fbaa0004d90 2023-11-24T06:24:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa800f040 con 0x7fbaa0004d90 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.515+0000 7fbaaf7fe700 1 -- 172.21.15.114:0/1949481721 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8014480 con 0x7fbaa0004d90 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 -- 172.21.15.114:0/1949481721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaa0004d90 msgr2=0x7fbaa0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 --2- 172.21.15.114:0/1949481721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbaa0004d90 0x7fbaa0005210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009fc0 tx=0x7fbaa0005880 comp rx=0 tx=0).stop 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab57cc700 1 -- 172.21.15.114:0/1949481721 reap_dead start 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 -- 172.21.15.114:0/1949481721 shutdown_connections 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 -- 172.21.15.114:0/1949481721 >> 172.21.15.114:0/1949481721 conn(0x7fbab0071b90 msgr2=0x7fbab0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 -- 172.21.15.114:0/1949481721 shutdown_connections 2023-11-24T06:24:43.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.516+0000 7fbab722f700 1 -- 172.21.15.114:0/1949481721 wait complete. 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.517+0000 7fbab722f700 1 Processor -- start 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.517+0000 7fbab722f700 1 -- start start 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 0x7fbab00cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab0147050 0x7fbab00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0157a00 con 0x7fbab01f7310 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00cecf0 con 0x7fbab01f6f30 2023-11-24T06:24:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab722f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab01573b0 con 0x7fbab0147050 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbaaffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbaaffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38610/0 (socket says 172.21.15.114:38610) 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbaaffff700 1 -- 172.21.15.114:0/1283925360 learned_addr learned my addr 172.21.15.114:0/1283925360 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab4fcb700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 0x7fbab00cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.518+0000 7fbab57cc700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab0147050 0x7fbab00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 -- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab0147050 msgr2=0x7fbab00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab0147050 0x7fbab00ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 -- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 msgr2=0x7fbab00cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 0x7fbab00cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 -- 172.21.15.114:0/1283925360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0157690 con 0x7fbab01f7310 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbab4fcb700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 0x7fbab00cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaaffff700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00f7f0 tx=0x7fba9c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:43.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0052e0 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c005440 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab00ce570 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0038c0 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.519+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab00cea80 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.520+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab00753f0 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.520+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba9c003a20 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.521+0000 7fbaad7fa700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 0x7fba9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.521+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba9c0893f0 con 0x7fbab01f7310 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.521+0000 7fbab4fcb700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 0x7fba9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.521+0000 7fbab4fcb700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 0x7fba9406d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007dd0 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:43.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.525+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba9c0544a0 con 0x7fbab01f7310 2023-11-24T06:24:43.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.760+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbab00b59f0 con 0x7fbab01f7310 2023-11-24T06:24:43.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.760+0000 7fbaad7fa700 1 -- 172.21.15.114:0/1283925360 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba9c057ac0 con 0x7fbab01f7310 2023-11-24T06:24:43.764 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:24:43.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 msgr2=0x7fba9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 0x7fba9406d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007dd0 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2023-11-24T06:24:43.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 msgr2=0x7fbab00cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:43.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00f7f0 tx=0x7fba9c00fbb0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 shutdown_connections 2023-11-24T06:24:43.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba9406aed0 0x7fba9406d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbab0147050 0x7fbab00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab01f6f30 0x7fbab00cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 --2- 172.21.15.114:0/1283925360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbab01f7310 0x7fbab00cf730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:43.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.762+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 >> 172.21.15.114:0/1283925360 conn(0x7fbab0071b90 msgr2=0x7fbab00b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:43.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.763+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 shutdown_connections 2023-11-24T06:24:43.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:43.763+0000 7fbab722f700 1 -- 172.21.15.114:0/1283925360 wait complete. 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:24:43.778 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:24:43.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:43.779 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42396sec preferred_lft 42396sec 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c36d:1855:677d:d218/64 scope link noprefixroute 2023-11-24T06:24:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:43.802 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:24:43.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:24:43.802 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:44.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.063 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:24:44.090 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:24:44.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.092 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-11-24T06:24:44.164 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-11-24T06:24:44.165 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:24:44.165 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:24:44.165 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:24:44.165 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:24:44.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.166 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:44.166 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:44.166 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:24:44.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:24:44.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:24:44.167 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:44.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.402 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:24:44.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.420 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-11-24T06:24:44.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.477 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:44.477 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:44.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:24:44.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:24:44.478 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:44.572 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:24:44.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.572 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:44.572 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:44.572 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:24:44.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:24:44.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:24:44.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:24:44.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:24:44.573 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.784 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:44.784 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:44.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-11-24T06:24:44.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:24:44.784 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:44.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:44.886 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:24:44.906 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:24:44.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:44.906 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:24:44.964 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:24:44.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:44.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:45.069 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:45.070 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:45.179 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:45.179 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:45.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:24:45.257 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:24:45.259 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:24:45.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:45.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:24:45.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:45.286 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:45.361 INFO:teuthology.orchestra.run.smithi172.stdout:client4734 2023-11-24T06:24:45.362 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:24:45.362 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:45.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:45.420 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:45.492 INFO:teuthology.orchestra.run.smithi172.stdout:client4734 2023-11-24T06:24:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:45.493 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:45.565 INFO:teuthology.orchestra.run.smithi172.stdout:client4734 2023-11-24T06:24:45.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:45.566 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.client4734/status') 2023-11-24T06:24:45.645 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4734 (3)172.21.15.172:0/3884292580 2023-11-24T06:24:45.645 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2023-11-24T06:24:45.646 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2023-11-24T06:24:45.646 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2023-11-24T06:24:45.647 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 2.6261e-05 s, 2.7 MB/s 2023-11-24T06:24:45.647 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-24T06:24:45.647 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:24:45.647 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:24:45.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:24:45.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.780+0000 7f4af1d28700 1 Processor -- start 2023-11-24T06:24:45.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.780+0000 7f4af1d28700 1 -- start start 2023-11-24T06:24:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.781+0000 7f4af1d28700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4aec06c710 0x7f4aec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.781+0000 7f4af1d28700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 0x7f4aec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.781+0000 7f4af1d28700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0ae430 con 0x7f4aec15e080 2023-11-24T06:24:45.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.781+0000 7f4af1d28700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec0af0b0 con 0x7f4aec06c710 2023-11-24T06:24:45.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.781+0000 7f4af1d28700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0b05a0 con 0x7f4aec0af4d0 2023-11-24T06:24:45.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 0x7f4aec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:45.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeb7fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4aec15e080 0x7f4aec06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:47922/0 (socket says 172.21.15.114:47922) 2023-11-24T06:24:45.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeb7fe700 1 -- 172.21.15.114:0/3851996317 learned_addr learned my addr 172.21.15.114:0/3851996317 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4ae2ffd700 1 --2- 172.21.15.114:0/3851996317 >> v2:172.21.15.172:3300/0 conn(0x7f4aec06c710 0x7f4aec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 -- 172.21.15.114:0/3851996317 >> v2:172.21.15.172:3300/0 conn(0x7f4aec06c710 msgr2=0x7f4aec0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 --2- 172.21.15.114:0/3851996317 >> v2:172.21.15.172:3300/0 conn(0x7f4aec06c710 0x7f4aec0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 -- 172.21.15.114:0/3851996317 >> v1:172.21.15.172:6789/0 conn(0x7f4aec15e080 legacy=0x7f4aec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 -- 172.21.15.114:0/3851996317 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0ae710 con 0x7f4aec0af4d0 2023-11-24T06:24:45.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeaffd700 1 -- 172.21.15.114:0/3851996317 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040811400 0 0) 0x7f4aec0ae430 con 0x7f4aec15e080 2023-11-24T06:24:45.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aebfff700 1 --2- 172.21.15.114:0/3851996317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 0x7f4aec0afb90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6f08839f75147c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:45.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeaffd700 1 -- 172.21.15.114:0/3851996317 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400e070 con 0x7f4aec0af4d0 2023-11-24T06:24:45.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeaffd700 1 -- 172.21.15.114:0/3851996317 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400bbd0 con 0x7f4aec0af4d0 2023-11-24T06:24:45.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4aeaffd700 1 -- 172.21.15.114:0/3851996317 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4012430 con 0x7f4aec0af4d0 2023-11-24T06:24:45.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 -- 172.21.15.114:0/3851996317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 msgr2=0x7f4aec0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:45.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 --2- 172.21.15.114:0/3851996317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 0x7f4aec0afb90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009d90 tx=0x7f4ae400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:45.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 -- 172.21.15.114:0/3851996317 shutdown_connections 2023-11-24T06:24:45.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 --2- 172.21.15.114:0/3851996317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec0af4d0 0x7f4aec0afb90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:45.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 --2- 172.21.15.114:0/3851996317 >> v2:172.21.15.172:3300/0 conn(0x7f4aec06c710 0x7f4aec0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:45.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 -- 172.21.15.114:0/3851996317 >> 172.21.15.114:0/3851996317 conn(0x7f4aec071b90 msgr2=0x7f4aec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:45.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 -- 172.21.15.114:0/3851996317 shutdown_connections 2023-11-24T06:24:45.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.782+0000 7f4af1d28700 1 -- 172.21.15.114:0/3851996317 wait complete. 2023-11-24T06:24:45.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.784+0000 7f4af1d28700 1 Processor -- start 2023-11-24T06:24:45.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 -- start start 2023-11-24T06:24:45.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec06c710 0x7f4aec151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aec0af4d0 0x7f4aec160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec160810 con 0x7f4aec15e080 2023-11-24T06:24:45.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15fc00 con 0x7f4aec0af4d0 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4af1d28700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15fee0 con 0x7f4aec06c710 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38628/0 (socket says 172.21.15.114:38628) 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.785+0000 7f4aebfff700 1 -- 172.21.15.114:0/2434392796 learned_addr learned my addr 172.21.15.114:0/2434392796 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4ae2ffd700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aec0af4d0 0x7f4aec160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 -- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec06c710 msgr2=0x7f4aec151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec06c710 0x7f4aec151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 -- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aec0af4d0 msgr2=0x7f4aec160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aec0af4d0 0x7f4aec160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 -- 172.21.15.114:0/2434392796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4009a70 con 0x7f4aec15e080 2023-11-24T06:24:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4aebfff700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4aec09f040 tx=0x7f4ae4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4012430 con 0x7f4aec15e080 2023-11-24T06:24:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec1601c0 con 0x7f4aec15e080 2023-11-24T06:24:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4020070 con 0x7f4aec15e080 2023-11-24T06:24:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4004280 con 0x7f4aec15e080 2023-11-24T06:24:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.786+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec151b40 con 0x7f4aec15e080 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.787+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0753f0 con 0x7f4aec15e080 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.788+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ae40043e0 con 0x7f4aec15e080 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.788+0000 7f4ae8ff9700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 0x7f4acc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.788+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ae4022070 con 0x7f4aec15e080 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.788+0000 7f4aeb7fe700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 0x7f4acc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.788+0000 7f4aeb7fe700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 0x7f4acc06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0b2930 tx=0x7f4adc0079f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:45.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:45.792+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ae40567c0 con 0x7f4aec15e080 2023-11-24T06:24:46.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.027+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4aec160950 con 0x7f4aec15e080 2023-11-24T06:24:46.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.027+0000 7f4ae8ff9700 1 -- 172.21.15.114:0/2434392796 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4aec160950 con 0x7f4aec15e080 2023-11-24T06:24:46.030 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:24:46.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 msgr2=0x7f4acc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:46.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 0x7f4acc06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0b2930 tx=0x7f4adc0079f0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 msgr2=0x7f4aec15a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:46.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4aec09f040 tx=0x7f4ae4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 shutdown_connections 2023-11-24T06:24:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4acc06aed0 0x7f4acc06d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aec15e080 0x7f4aec15a720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aec06c710 0x7f4aec151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 --2- 172.21.15.114:0/2434392796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aec0af4d0 0x7f4aec160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 >> 172.21.15.114:0/2434392796 conn(0x7f4aec071b90 msgr2=0x7f4aec0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 shutdown_connections 2023-11-24T06:24:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:46.029+0000 7f4af1d28700 1 -- 172.21.15.114:0/2434392796 wait complete. 2023-11-24T06:24:46.043 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:24:46.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:24:46.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-11-24T06:24:46.043 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:24:46.044 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:24:46.044 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:24:46.044 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:24:46.044 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:24:46.044 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:24:46.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.044 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-11-24T06:24:46.062 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:46.062 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:46.063 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:46.064 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:46.064 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:46.064 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:24:46.064 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:46.064 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:24:46.065 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:46.065 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:24:46.065 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42394sec preferred_lft 42394sec 2023-11-24T06:24:46.065 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c36d:1855:677d:d218/64 scope link noprefixroute 2023-11-24T06:24:46.065 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.066 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:46.066 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:24:46.066 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:24:46.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.067 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::4cb3:adff:fe77:17e6/64 scope link 2023-11-24T06:24:46.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.067 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:24:46.067 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:24:46.068 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::3838:94ff:fea3:1687/64 scope link tentative 2023-11-24T06:24:46.068 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.068 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:24:46.128 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:24:46.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.129 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-11-24T06:24:46.179 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:24:46.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:24:46.188 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-11-24T06:24:46.189 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:24:46.189 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:24:46.189 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:24:46.189 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:24:46.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:24:46.190 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:46.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.295 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:24:46.351 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:24:46.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.352 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2023-11-24T06:24:46.408 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:24:46.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.408 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:46.408 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:46.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:24:46.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:24:46.409 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:46.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.502 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.539 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:46.540 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 52:ba:e4:26:0c:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:24:46.540 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:24:46.540 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.540 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::50ba:e4ff:fe26:c15/64 scope link 2023-11-24T06:24:46.540 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.558 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:24:46.641 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:24:46.641 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:24:46.641 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:24:46.641 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.641 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 52:ba:e4:26:0c:15 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::50ba:e4ff:fe26:c15/64 scope link 2023-11-24T06:24:46.642 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:24:46.649 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:24:46.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:24:46.650 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:46.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:46.855 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:24:46.855 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:24:46.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2023-11-24T06:24:46.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:24:46.855 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:24:46.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:46.973 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:24:46.995 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:24:46.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:24:46.995 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:24:47.053 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-24T06:24:47.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:47.139 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:47.139 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:47.165 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:47.165 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:47.165 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:47.166 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 3@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:47.166 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:47.166 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:47.166 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:47.166 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:47.244 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:24:47.244 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:24:47.245 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:24:47.245 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 3@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:24:47.245 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:24:47.245 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:24:47.245 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:24:47.246 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:24:47.246 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:24:47.246 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:24:47.246 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:24:47.247 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:24:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:24:47.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.275 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:47.347 INFO:teuthology.orchestra.run.smithi172.stdout:client4740 2023-11-24T06:24:47.348 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:24:47.348 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:47.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.407 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:47.479 INFO:teuthology.orchestra.run.smithi172.stdout:client4740 2023-11-24T06:24:47.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.479 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:24:47.552 INFO:teuthology.orchestra.run.smithi172.stdout:client4740 2023-11-24T06:24:47.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:47.552 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.client4740/status') 2023-11-24T06:24:47.635 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4740 (3)172.21.15.172:0/698307423 2023-11-24T06:24:47.636 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2023-11-24T06:24:47.636 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2023-11-24T06:24:47.636 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2023-11-24T06:24:47.637 INFO:teuthology.orchestra.run.smithi172.stderr:70 bytes copied, 3.8063e-05 s, 1.8 MB/s 2023-11-24T06:24:47.637 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:24:47.653 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.785+0000 7effa64e9700 1 Processor -- start 2023-11-24T06:24:47.788 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.785+0000 7effa64e9700 1 -- start start 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa64e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa64e9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effa01624d0 0x7effa0162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa64e9700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa006dd80 con 0x7effa00afe70 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa64e9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa015f880 con 0x7effa01624d0 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa64e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa015ed90 con 0x7effa015bef0 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effa01624d0 0x7effa0162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effa01624d0 0x7effa0162960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38632/0 (socket says 172.21.15.114:38632) 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7eff9f7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7effa00afe70 0x7effa00b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54340/0 (socket says 172.21.15.114:54340) 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 -- 172.21.15.114:0/1350579588 learned_addr learned my addr 172.21.15.114:0/1350579588 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7eff9ffff700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 -- 172.21.15.114:0/1350579588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 msgr2=0x7effa015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 -- 172.21.15.114:0/1350579588 >> v1:172.21.15.114:6789/0 conn(0x7effa00afe70 legacy=0x7effa00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 -- 172.21.15.114:0/1350579588 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa015c2d0 con 0x7effa01624d0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7eff9effd700 1 -- 172.21.15.114:0/1350579588 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969743512 0 0) 0x7effa006dd80 con 0x7effa00afe70 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.786+0000 7effa4a86700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0162960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff98009e50 tx=0x7eff9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55166a2390cc36a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7eff9effd700 1 -- 172.21.15.114:0/1350579588 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800e070 con 0x7effa01624d0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7eff9effd700 1 -- 172.21.15.114:0/1350579588 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800b910 con 0x7effa01624d0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7eff9effd700 1 -- 172.21.15.114:0/1350579588 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800bcc0 con 0x7effa01624d0 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 -- 172.21.15.114:0/1350579588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 msgr2=0x7effa0162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0162960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff98009e50 tx=0x7eff9800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 -- 172.21.15.114:0/1350579588 shutdown_connections 2023-11-24T06:24:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 --2- 172.21.15.114:0/1350579588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0162960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 -- 172.21.15.114:0/1350579588 >> 172.21.15.114:0/1350579588 conn(0x7effa0071ac0 msgr2=0x7effa0071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 -- 172.21.15.114:0/1350579588 shutdown_connections 2023-11-24T06:24:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.787+0000 7effa64e9700 1 -- 172.21.15.114:0/1350579588 wait complete. 2023-11-24T06:24:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.789+0000 7effa64e9700 1 Processor -- start 2023-11-24T06:24:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.789+0000 7effa64e9700 1 -- start start 2023-11-24T06:24:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effa00afe70 0x7effa015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa015dc60 con 0x7effa01624d0 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa015df40 con 0x7effa00afe70 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa64e9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa015e220 con 0x7effa015bef0 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38642/0 (socket says 172.21.15.114:38642) 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 -- 172.21.15.114:0/167614549 learned_addr learned my addr 172.21.15.114:0/167614549 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7eff9f7fe700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7eff9ffff700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effa00afe70 0x7effa015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 -- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 msgr2=0x7effa0154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa0154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 -- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effa00afe70 msgr2=0x7effa015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effa00afe70 0x7effa015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 -- 172.21.15.114:0/167614549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98009b30 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7effa4a86700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b680 tx=0x7eff98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.790+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980099b0 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.791+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980092e0 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.791+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9801c660 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.791+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa015e500 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.791+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa015d950 con 0x7effa01624d0 2023-11-24T06:24:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.792+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa0067f60 con 0x7effa01624d0 2023-11-24T06:24:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.792+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff9801c7c0 con 0x7effa01624d0 2023-11-24T06:24:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.792+0000 7eff9cff9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.793+0000 7eff9ffff700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.793+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff9808b2c0 con 0x7effa01624d0 2023-11-24T06:24:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.793+0000 7eff9ffff700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 0x7eff8c06d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7effa0157110 tx=0x7eff94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:47.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.797+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff98056400 con 0x7effa01624d0 2023-11-24T06:24:47.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.987+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effa00b41b0 con 0x7effa01624d0 2023-11-24T06:24:47.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.988+0000 7eff9cff9700 1 -- 172.21.15.114:0/167614549 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7eff98059a20 con 0x7effa01624d0 2023-11-24T06:24:47.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:47.991 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 msgr2=0x7eff8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 0x7eff8c06d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7effa0157110 tx=0x7eff94008040 comp rx=0 tx=0).stop 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 msgr2=0x7effa0157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff9800b680 tx=0x7eff98004ad0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 shutdown_connections 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effa01624d0 0x7effa0157160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effa00afe70 0x7effa015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 --2- 172.21.15.114:0/167614549 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effa015bef0 0x7effa0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.990+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 >> 172.21.15.114:0/167614549 conn(0x7effa0071ac0 msgr2=0x7effa00cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.991+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 shutdown_connections 2023-11-24T06:24:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:47.991+0000 7effa64e9700 1 -- 172.21.15.114:0/167614549 wait complete. 2023-11-24T06:24:48.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:48.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:48.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 Processor -- start 2023-11-24T06:24:48.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 -- start start 2023-11-24T06:24:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f55040746a0 0x7f55040b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040b3c60 con 0x7f55040afc90 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040b58c0 con 0x7f550415e160 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.128+0000 7f550a974700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55040b4500 con 0x7f55040746a0 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5508f11700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f55040afc90 0x7f550406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54354/0 (socket says 172.21.15.114:54354) 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5508f11700 1 -- 172.21.15.114:0/1618908949 learned_addr learned my addr 172.21.15.114:0/1618908949 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f55037fe700 1 --2- 172.21.15.114:0/1618908949 >> v2:172.21.15.172:3300/0 conn(0x7f55040746a0 0x7f55040b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451386038 0 0) 0x7f55040b3c60 con 0x7f55040afc90 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec0035b0 con 0x7f55040afc90 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868454791 0 0) 0x7f55040b58c0 con 0x7f550415e160 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55040b3c60 con 0x7f550415e160 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f55037fe700 1 -- 172.21.15.114:0/1618908949 >> v1:172.21.15.172:6789/0 conn(0x7f550415e160 legacy=0x7f5504073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f55037fe700 1 -- 172.21.15.114:0/1618908949 >> v1:172.21.15.114:6789/0 conn(0x7f55040afc90 legacy=0x7f550406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f55037fe700 1 -- 172.21.15.114:0/1618908949 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550406bdc0 con 0x7f55040746a0 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f55037fe700 1 --2- 172.21.15.114:0/1618908949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b5da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009840 tx=0x7f54f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12685e9eba970b7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4011830 con 0x7f55040746a0 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f4011990 con 0x7f55040746a0 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.129+0000 7f5502ffd700 1 -- 172.21.15.114:0/1618908949 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4011d20 con 0x7f55040746a0 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 -- 172.21.15.114:0/1618908949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 msgr2=0x7f55040b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 --2- 172.21.15.114:0/1618908949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b5da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009840 tx=0x7f54f4009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 -- 172.21.15.114:0/1618908949 shutdown_connections 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 --2- 172.21.15.114:0/1618908949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b5da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 -- 172.21.15.114:0/1618908949 >> 172.21.15.114:0/1618908949 conn(0x7f55040d0bf0 msgr2=0x7f55040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 -- 172.21.15.114:0/1618908949 shutdown_connections 2023-11-24T06:24:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.130+0000 7f550a974700 1 -- 172.21.15.114:0/1618908949 wait complete. 2023-11-24T06:24:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.131+0000 7f550a974700 1 Processor -- start 2023-11-24T06:24:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 -- start start 2023-11-24T06:24:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 0x7f5504161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550415f860 con 0x7f550415e160 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040b2640 con 0x7f55040746a0 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f550a974700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550415ec50 con 0x7f55040afc90 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.132+0000 7f5508f11700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38654/0 (socket says 172.21.15.114:38654) 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 -- 172.21.15.114:0/3226287029 learned_addr learned my addr 172.21.15.114:0/3226287029 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f55037fe700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 0x7f5504161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5503fff700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 -- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 msgr2=0x7f5504161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 0x7f5504161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 -- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 msgr2=0x7f55040b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 -- 172.21.15.114:0/3226287029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40094a0 con 0x7f550415e160 2023-11-24T06:24:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f55037fe700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 0x7f5504161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5508f11700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00cba0 tx=0x7f54fc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00ab20 con 0x7f550415e160 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc00ac80 con 0x7f550415e160 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550415ef30 con 0x7f550415e160 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc012400 con 0x7f550415e160 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.133+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550415f4f0 con 0x7f550415e160 2023-11-24T06:24:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.134+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5504153c70 con 0x7f550415e160 2023-11-24T06:24:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.134+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54fc01b3b0 con 0x7f550415e160 2023-11-24T06:24:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.135+0000 7f5500ff9700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 0x7f54f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.135+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54fc0891b0 con 0x7f550415e160 2023-11-24T06:24:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.135+0000 7f5503fff700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 0x7f54f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.135+0000 7f5503fff700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 0x7f54f006d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.139+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54fc0543a0 con 0x7f550415e160 2023-11-24T06:24:48.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.329+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5504068fb0 con 0x7f550415e160 2023-11-24T06:24:48.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.330+0000 7f5500ff9700 1 -- 172.21.15.114:0/3226287029 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54fc0579c0 con 0x7f550415e160 2023-11-24T06:24:48.332 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:48.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.331+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 msgr2=0x7f54f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.331+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 0x7f54f006d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f8009040 comp rx=0 tx=0).stop 2023-11-24T06:24:48.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.331+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 msgr2=0x7f55040b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.331+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00cba0 tx=0x7f54fc00cf60 comp rx=0 tx=0).stop 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 shutdown_connections 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f54f006aed0 0x7f54f006d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f550415e160 0x7f55040b02f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55040746a0 0x7f55040b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 --2- 172.21.15.114:0/3226287029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55040afc90 0x7f5504161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 >> 172.21.15.114:0/3226287029 conn(0x7f55040d0bf0 msgr2=0x7f550406ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 shutdown_connections 2023-11-24T06:24:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.332+0000 7f550a974700 1 -- 172.21.15.114:0/3226287029 wait complete. 2023-11-24T06:24:48.346 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:48.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.466+0000 7feb36b2e700 1 Processor -- start 2023-11-24T06:24:48.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.467+0000 7feb36b2e700 1 -- start start 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb36b2e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb300c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb36b2e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb300af5c0 0x7feb3015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb36b2e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300afcc0 con 0x7feb3015ac30 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb36b2e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300b0560 con 0x7feb300af5c0 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb36b2e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300c5a20 con 0x7feb300c5100 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb300af5c0 0x7feb3015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb350cb700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7feb3015ac30 0x7feb3015b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54372/0 (socket says 172.21.15.114:54372) 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb300af5c0 0x7feb3015a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38662/0 (socket says 172.21.15.114:38662) 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb350cb700 1 -- 172.21.15.114:0/3958757731 learned_addr learned my addr 172.21.15.114:0/3958757731 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb348ca700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb300c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 -- 172.21.15.114:0/3958757731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 msgr2=0x7feb300c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb300c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 -- 172.21.15.114:0/3958757731 >> v1:172.21.15.114:6789/0 conn(0x7feb3015ac30 legacy=0x7feb3015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 -- 172.21.15.114:0/3958757731 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3015e9f0 con 0x7feb300af5c0 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2f7fe700 1 -- 172.21.15.114:0/3958757731 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976630858 0 0) 0x7feb300afcc0 con 0x7feb3015ac30 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2ffff700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb3015a6f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009840 tx=0x7feb1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d33d0c634d0aef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2f7fe700 1 -- 172.21.15.114:0/3958757731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c011830 con 0x7feb300af5c0 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2f7fe700 1 -- 172.21.15.114:0/3958757731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c011990 con 0x7feb300af5c0 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.468+0000 7feb2f7fe700 1 -- 172.21.15.114:0/3958757731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c011d20 con 0x7feb300af5c0 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 -- 172.21.15.114:0/3958757731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 msgr2=0x7feb3015a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb3015a6f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009840 tx=0x7feb1c009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 -- 172.21.15.114:0/3958757731 shutdown_connections 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb300c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3958757731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb3015a6f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 -- 172.21.15.114:0/3958757731 >> 172.21.15.114:0/3958757731 conn(0x7feb300b1e90 msgr2=0x7feb300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 -- 172.21.15.114:0/3958757731 shutdown_connections 2023-11-24T06:24:48.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.469+0000 7feb36b2e700 1 -- 172.21.15.114:0/3958757731 wait complete. 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.470+0000 7feb36b2e700 1 Processor -- start 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.471+0000 7feb36b2e700 1 -- start start 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb30153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3015ac30 0x7feb301494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb301543e0 con 0x7feb300af5c0 2023-11-24T06:24:48.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb301499f0 con 0x7feb3015ac30 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3014a2e0 con 0x7feb300c5100 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38666/0 (socket says 172.21.15.114:38666) 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 -- 172.21.15.114:0/3860892970 learned_addr learned my addr 172.21.15.114:0/3860892970 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb2ffff700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb30153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb350cb700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3015ac30 0x7feb301494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 -- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 msgr2=0x7feb30153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb30153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 -- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3015ac30 msgr2=0x7feb301494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3015ac30 0x7feb301494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 -- 172.21.15.114:0/3860892970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c0094a0 con 0x7feb300af5c0 2023-11-24T06:24:48.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb348ca700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e8a0 tx=0x7feb2400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24009590 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24013680 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3014a5c0 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240139f0 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.472+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3014ab80 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.473+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3014b1f0 con 0x7feb300af5c0 2023-11-24T06:24:48.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.474+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb24013b50 con 0x7feb300af5c0 2023-11-24T06:24:48.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.474+0000 7feb2d7fa700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 0x7feb1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.474+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb24089450 con 0x7feb300af5c0 2023-11-24T06:24:48.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.474+0000 7feb2ffff700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 0x7feb1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.475+0000 7feb2ffff700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 0x7feb1406d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009fd0 tx=0x7feb1c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.479+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb240544f0 con 0x7feb300af5c0 2023-11-24T06:24:48.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.667+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb30068fb0 con 0x7feb300af5c0 2023-11-24T06:24:48.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.668+0000 7feb2d7fa700 1 -- 172.21.15.114:0/3860892970 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb24057b10 con 0x7feb300af5c0 2023-11-24T06:24:48.670 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:48.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 msgr2=0x7feb1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 0x7feb1406d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009fd0 tx=0x7feb1c01c040 comp rx=0 tx=0).stop 2023-11-24T06:24:48.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 msgr2=0x7feb30149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e8a0 tx=0x7feb2400ebb0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 shutdown_connections 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feb1406aed0 0x7feb1406d390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3015ac30 0x7feb301494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb300af5c0 0x7feb30149be0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 --2- 172.21.15.114:0/3860892970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300c5100 0x7feb30153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 >> 172.21.15.114:0/3860892970 conn(0x7feb300b1e90 msgr2=0x7feb300b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 shutdown_connections 2023-11-24T06:24:48.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.670+0000 7feb36b2e700 1 -- 172.21.15.114:0/3860892970 wait complete. 2023-11-24T06:24:48.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:48.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.805+0000 7f63d74db700 1 Processor -- start 2023-11-24T06:24:48.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.805+0000 7f63d74db700 1 -- start start 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d74db700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63d015e160 0x7f63d00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d74db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63d0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d74db700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00b3c60 con 0x7f63d00744d0 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d74db700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00b4500 con 0x7f63d015e160 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d74db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0074960 con 0x7f63d00afc90 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d5a78700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f63d00744d0 0x7f63d006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54386/0 (socket says 172.21.15.114:54386) 2023-11-24T06:24:48.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d5a78700 1 -- 172.21.15.114:0/475160786 learned_addr learned my addr 172.21.15.114:0/475160786 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d5277700 1 --2- 172.21.15.114:0/475160786 >> v2:172.21.15.172:3300/0 conn(0x7f63d015e160 0x7f63d00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63d4a76700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63d0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17794355 0 0) 0x7f63d00b3c60 con 0x7f63d00744d0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63bc0035b0 con 0x7f63d00744d0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1069822993 0 0) 0x7f63bc0035b0 con 0x7f63d00744d0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 msgr2=0x7f63d0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63d0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 >> v2:172.21.15.172:3300/0 conn(0x7f63d015e160 msgr2=0x7f63d00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 --2- 172.21.15.114:0/475160786 >> v2:172.21.15.172:3300/0 conn(0x7f63d015e160 0x7f63d00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.806+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00b58c0 con 0x7f63d00744d0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f63c80019f0 con 0x7f63d00744d0 2023-11-24T06:24:48.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 >> v1:172.21.15.114:6789/0 conn(0x7f63d00744d0 legacy=0x7f63d006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63bc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63bc003f80 0x7f63bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc004d90 0x7f63bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00b4500 con 0x7f63bc003f80 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d0074960 con 0x7f63bc004d90 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc00ba10 con 0x7f63d00afc90 2023-11-24T06:24:48.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63bc003f80 0x7f63bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d5277700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63bc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d5a78700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc004d90 0x7f63bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 -- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 msgr2=0x7f63bc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00afc90 0x7f63bc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 -- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc004d90 msgr2=0x7f63bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc004d90 0x7f63bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 -- 172.21.15.114:0/475160786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0035b0 con 0x7f63bc003f80 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d5a78700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc004d90 0x7f63bc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63d4a76700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63bc003f80 0x7f63bc004850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009b60 tx=0x7f63bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9e941f75a5bf4c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000e660 con 0x7f63bc003f80 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c000e7c0 con 0x7f63bc003f80 2023-11-24T06:24:48.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.807+0000 7f63c7fff700 1 -- 172.21.15.114:0/475160786 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000eb30 con 0x7f63bc003f80 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 -- 172.21.15.114:0/475160786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63bc003f80 msgr2=0x7f63bc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 --2- 172.21.15.114:0/475160786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63bc003f80 0x7f63bc004850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009b60 tx=0x7f63bc005880 comp rx=0 tx=0).stop 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d5a78700 1 -- 172.21.15.114:0/475160786 reap_dead start 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 -- 172.21.15.114:0/475160786 shutdown_connections 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 -- 172.21.15.114:0/475160786 >> 172.21.15.114:0/475160786 conn(0x7f63d00d0bf0 msgr2=0x7f63d00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 -- 172.21.15.114:0/475160786 shutdown_connections 2023-11-24T06:24:48.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.808+0000 7f63d74db700 1 -- 172.21.15.114:0/475160786 wait complete. 2023-11-24T06:24:48.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.809+0000 7f63d74db700 1 Processor -- start 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.809+0000 7f63d74db700 1 -- start start 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc003f80 0x7f63d00c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00c6620 0x7f63d01f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00c6200 con 0x7f63d01f3d80 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00c5bd0 con 0x7f63bc003f80 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d74db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00ca500 con 0x7f63d00c6620 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38690/0 (socket says 172.21.15.114:38690) 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 -- 172.21.15.114:0/4245802912 learned_addr learned my addr 172.21.15.114:0/4245802912 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:48.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5277700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc003f80 0x7f63d00c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d4a76700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00c6620 0x7f63d01f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 -- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00c6620 msgr2=0x7f63d01f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00c6620 0x7f63d01f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 -- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc003f80 msgr2=0x7f63d00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc003f80 0x7f63d00c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 -- 172.21.15.114:0/4245802912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d01f4330 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63d5a78700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63d009ef70 tx=0x7f63c8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.810+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800ed10 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.811+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c800ee90 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.811+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8013660 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.811+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d01f4610 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.811+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d00c71d0 con 0x7f63d01f3d80 2023-11-24T06:24:48.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.812+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d01f4800 con 0x7f63d01f3d80 2023-11-24T06:24:48.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.812+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63c80137c0 con 0x7f63d01f3d80 2023-11-24T06:24:48.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.812+0000 7f63c5ffb700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 0x7f63b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:48.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.812+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c8089b70 con 0x7f63d01f3d80 2023-11-24T06:24:48.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.812+0000 7f63d5277700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 0x7f63b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:48.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.813+0000 7f63d5277700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 0x7f63b406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63d00d0850 tx=0x7f63cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:48.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:48.817+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63c8054cb0 con 0x7f63d01f3d80 2023-11-24T06:24:49.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.009+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63d00c6340 con 0x7f63d01f3d80 2023-11-24T06:24:49.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.009+0000 7f63c5ffb700 1 -- 172.21.15.114:0/4245802912 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63d00c6340 con 0x7f63d01f3d80 2023-11-24T06:24:49.012 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:49.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 msgr2=0x7f63b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 0x7f63b406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f63d00d0850 tx=0x7f63cc008040 comp rx=0 tx=0).stop 2023-11-24T06:24:49.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 msgr2=0x7f63d00b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63d009ef70 tx=0x7f63c8001ae0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 shutdown_connections 2023-11-24T06:24:49.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f63b406aed0 0x7f63b406d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63d01f3d80 0x7f63d00b1710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63bc003f80 0x7f63d00c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 --2- 172.21.15.114:0/4245802912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63d00c6620 0x7f63d01f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 >> 172.21.15.114:0/4245802912 conn(0x7f63d00d0bf0 msgr2=0x7f63d015d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 shutdown_connections 2023-11-24T06:24:49.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.012+0000 7f63d74db700 1 -- 172.21.15.114:0/4245802912 wait complete. 2023-11-24T06:24:49.028 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:49.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.145+0000 7f407050b700 1 Processor -- start 2023-11-24T06:24:49.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.145+0000 7f407050b700 1 -- start start 2023-11-24T06:24:49.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.146+0000 7f407050b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 0x7f40680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.146+0000 7f407050b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f40680af4d0 0x7f40680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.146+0000 7f407050b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680ae430 con 0x7f406806c000 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.146+0000 7f407050b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40680af0b0 con 0x7f40680af4d0 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.146+0000 7f407050b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40680b05a0 con 0x7f406815e080 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f40680af4d0 0x7f40680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f40680af4d0 0x7f40680afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38694/0 (socket says 172.21.15.114:38694) 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 -- 172.21.15.114:0/1014481431 learned_addr learned my addr 172.21.15.114:0/1014481431 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406e2a7700 1 --2- 172.21.15.114:0/1014481431 >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 0x7f40680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 -- 172.21.15.114:0/1014481431 >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 msgr2=0x7f40680b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 --2- 172.21.15.114:0/1014481431 >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 0x7f40680b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 -- 172.21.15.114:0/1014481431 >> v1:172.21.15.172:6789/0 conn(0x7f406806c000 legacy=0x7f4068073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 -- 172.21.15.114:0/1014481431 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680ae710 con 0x7f40680af4d0 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406e2a7700 1 --2- 172.21.15.114:0/1014481431 >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 0x7f40680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406daa6700 1 --2- 172.21.15.114:0/1014481431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40680af4d0 0x7f40680afb90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4068072a90 tx=0x7f4064009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f96ac233f418b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406d2a5700 1 -- 172.21.15.114:0/1014481431 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064004230 con 0x7f40680af4d0 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406d2a5700 1 -- 172.21.15.114:0/1014481431 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4064004390 con 0x7f40680af4d0 2023-11-24T06:24:49.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.147+0000 7f406d2a5700 1 -- 172.21.15.114:0/1014481431 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640036a0 con 0x7f40680af4d0 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 -- 172.21.15.114:0/1014481431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40680af4d0 msgr2=0x7f40680afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 --2- 172.21.15.114:0/1014481431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40680af4d0 0x7f40680afb90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4068072a90 tx=0x7f4064009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 -- 172.21.15.114:0/1014481431 shutdown_connections 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 --2- 172.21.15.114:0/1014481431 >> v2:172.21.15.172:3300/0 conn(0x7f406815e080 0x7f40680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 --2- 172.21.15.114:0/1014481431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40680af4d0 0x7f40680afb90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 -- 172.21.15.114:0/1014481431 >> 172.21.15.114:0/1014481431 conn(0x7f4068071b90 msgr2=0x7f4068071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 -- 172.21.15.114:0/1014481431 shutdown_connections 2023-11-24T06:24:49.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.148+0000 7f407050b700 1 -- 172.21.15.114:0/1014481431 wait complete. 2023-11-24T06:24:49.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.149+0000 7f407050b700 1 Processor -- start 2023-11-24T06:24:49.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.149+0000 7f407050b700 1 -- start start 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 0x7f4068160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40680af4d0 0x7f4068154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068155000 con 0x7f406815e080 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068153a20 con 0x7f40680af4d0 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4068153d00 con 0x7f406806c000 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38704/0 (socket says 172.21.15.114:38704) 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 -- 172.21.15.114:0/2095042420 learned_addr learned my addr 172.21.15.114:0/2095042420 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406e2a7700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 0x7f4068160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406daa6700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40680af4d0 0x7f4068154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 -- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 msgr2=0x7f4068160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 0x7f4068160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 -- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40680af4d0 msgr2=0x7f4068154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40680af4d0 0x7f4068154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 -- 172.21.15.114:0/2095042420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064009550 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406e2a7700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 0x7f4068160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f406eaa8700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f406000cc10 tx=0x7f406000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000ab10 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000ac70 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4068153fe0 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060012420 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.150+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40681545a0 con 0x7f406815e080 2023-11-24T06:24:49.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.151+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40680753f0 con 0x7f406815e080 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.152+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4060012580 con 0x7f406815e080 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.152+0000 7f405affd700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 0x7f404c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.152+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40600898f0 con 0x7f406815e080 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.152+0000 7f406e2a7700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 0x7f404c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.152+0000 7f406e2a7700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 0x7f404c06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f405c006fd0 tx=0x7f405c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.156+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40600549a0 con 0x7f406815e080 2023-11-24T06:24:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.348+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4068072cf0 con 0x7f406815e080 2023-11-24T06:24:49.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.349+0000 7f405affd700 1 -- 172.21.15.114:0/2095042420 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f4060057fc0 con 0x7f406815e080 2023-11-24T06:24:49.352 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:49.352 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 msgr2=0x7f404c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 0x7f404c06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f405c006fd0 tx=0x7f405c008040 comp rx=0 tx=0).stop 2023-11-24T06:24:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 msgr2=0x7f4068153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f406000cc10 tx=0x7f406000cf20 comp rx=0 tx=0).stop 2023-11-24T06:24:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 shutdown_connections 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f404c06aed0 0x7f404c06d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f406815e080 0x7f4068153160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406806c000 0x7f4068160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 --2- 172.21.15.114:0/2095042420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40680af4d0 0x7f4068154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.351+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 >> 172.21.15.114:0/2095042420 conn(0x7f4068071b90 msgr2=0x7f4068074210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.352+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 shutdown_connections 2023-11-24T06:24:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.352+0000 7f407050b700 1 -- 172.21.15.114:0/2095042420 wait complete. 2023-11-24T06:24:49.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:49.367 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:49.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 Processor -- start 2023-11-24T06:24:49.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 -- start start 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f61a40d45d0 0x7f61a40b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40d2930 con 0x7f61a415e140 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40d2c10 con 0x7f61a40d39d0 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.488+0000 7f61a977e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a415e520 con 0x7f61a40d45d0 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f61a40d45d0 0x7f61a40b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a2ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f61a415e140 0x7f61a40d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54416/0 (socket says 172.21.15.114:54416) 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a2ffd700 1 -- 172.21.15.114:0/86216158 learned_addr learned my addr 172.21.15.114:0/86216158 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f61a40d45d0 0x7f61a40b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38708/0 (socket says 172.21.15.114:38708) 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 -- 172.21.15.114:0/86216158 >> v1:172.21.15.172:6789/0 conn(0x7f61a40d39d0 legacy=0x7f61a40b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:49.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 -- 172.21.15.114:0/86216158 >> v1:172.21.15.114:6789/0 conn(0x7f61a415e140 legacy=0x7f61a40d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 -- 172.21.15.114:0/86216158 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40d2ef0 con 0x7f61a40d45d0 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a1ffb700 1 -- 172.21.15.114:0/86216158 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641083217 0 0) 0x7f61a40d2930 con 0x7f61a415e140 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a27fc700 1 --2- 172.21.15.114:0/86216158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d45d0 0x7f61a40b70d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6198009840 tx=0x7f6198009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fa76bcb92c8813 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a1ffb700 1 -- 172.21.15.114:0/86216158 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198004230 con 0x7f61a40d45d0 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a1ffb700 1 -- 172.21.15.114:0/86216158 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198004390 con 0x7f61a40d45d0 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.489+0000 7f61a1ffb700 1 -- 172.21.15.114:0/86216158 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61980036a0 con 0x7f61a40d45d0 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 -- 172.21.15.114:0/86216158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d45d0 msgr2=0x7f61a40b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 --2- 172.21.15.114:0/86216158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d45d0 0x7f61a40b70d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6198009840 tx=0x7f6198009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 -- 172.21.15.114:0/86216158 shutdown_connections 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 --2- 172.21.15.114:0/86216158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d45d0 0x7f61a40b70d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 -- 172.21.15.114:0/86216158 >> 172.21.15.114:0/86216158 conn(0x7f61a406b410 msgr2=0x7f61a406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 -- 172.21.15.114:0/86216158 shutdown_connections 2023-11-24T06:24:49.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.490+0000 7f61a977e700 1 -- 172.21.15.114:0/86216158 wait complete. 2023-11-24T06:24:49.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.491+0000 7f61a977e700 1 Processor -- start 2023-11-24T06:24:49.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.491+0000 7f61a977e700 1 -- start start 2023-11-24T06:24:49.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 0x7f61a40cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a415e140 0x7f61a4151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c9150 con 0x7f61a40d39d0 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c8820 con 0x7f61a40d45d0 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a977e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c8b00 con 0x7f61a415e140 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38712/0 (socket says 172.21.15.114:38712) 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 -- 172.21.15.114:0/45395032 learned_addr learned my addr 172.21.15.114:0/45395032 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a27fc700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 0x7f61a40cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a37fe700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a415e140 0x7f61a4151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 -- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a415e140 msgr2=0x7f61a4151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a415e140 0x7f61a4151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 -- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 msgr2=0x7f61a40cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 0x7f61a40cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 -- 172.21.15.114:0/45395032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61980094a0 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a27fc700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 0x7f61a40cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.492+0000 7f61a2ffd700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f619400c550 tx=0x7f619400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.493+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194004c80 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.493+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194004e00 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.493+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40c8de0 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.493+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194007310 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.493+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40ce360 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.494+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a40cacc0 con 0x7f61a40d39d0 2023-11-24T06:24:49.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.494+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6194007470 con 0x7f61a40d39d0 2023-11-24T06:24:49.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.494+0000 7f618b7fe700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 0x7f619006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.495+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6194089360 con 0x7f61a40d39d0 2023-11-24T06:24:49.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.495+0000 7f61a27fc700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 0x7f619006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.495+0000 7f61a27fc700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 0x7f619006d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ce7e0 tx=0x7f6198008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.499+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6194054380 con 0x7f61a40d39d0 2023-11-24T06:24:49.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.687+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61a40d40f0 con 0x7f61a40d39d0 2023-11-24T06:24:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.688+0000 7f618b7fe700 1 -- 172.21.15.114:0/45395032 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61940579a0 con 0x7f61a40d39d0 2023-11-24T06:24:49.690 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:49.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.689+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 msgr2=0x7f619006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.689+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 0x7f619006d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ce7e0 tx=0x7f6198008030 comp rx=0 tx=0).stop 2023-11-24T06:24:49.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 msgr2=0x7f61a41609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f619400c550 tx=0x7f619400c910 comp rx=0 tx=0).stop 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 shutdown_connections 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f619006aed0 0x7f619006d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a415e140 0x7f61a4151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a40d39d0 0x7f61a41609b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 --2- 172.21.15.114:0/45395032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a40d45d0 0x7f61a40cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 >> 172.21.15.114:0/45395032 conn(0x7f61a406b410 msgr2=0x7f61a40b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 shutdown_connections 2023-11-24T06:24:49.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.690+0000 7f61a977e700 1 -- 172.21.15.114:0/45395032 wait complete. 2023-11-24T06:24:49.704 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.825+0000 7fc538963700 1 Processor -- start 2023-11-24T06:24:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.825+0000 7fc538963700 1 -- start start 2023-11-24T06:24:49.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc538963700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5341587f0 0x7fc53406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc538963700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 0x7fc53415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc538963700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53406b530 con 0x7fc53415b370 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc538963700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340ae5e0 con 0x7fc5341587f0 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc538963700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53415be30 con 0x7fc5340b4370 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5341587f0 0x7fc53406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc531d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc53415b370 0x7fc5340b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54428/0 (socket says 172.21.15.114:54428) 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc531d9b700 1 -- 172.21.15.114:0/1374760002 learned_addr learned my addr 172.21.15.114:0/1374760002 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc532d9d700 1 --2- 172.21.15.114:0/1374760002 >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 0x7fc53415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 -- 172.21.15.114:0/1374760002 >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 msgr2=0x7fc53415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 --2- 172.21.15.114:0/1374760002 >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 0x7fc53415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 -- 172.21.15.114:0/1374760002 >> v1:172.21.15.114:6789/0 conn(0x7fc53415b370 legacy=0x7fc5340b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 -- 172.21.15.114:0/1374760002 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53415c110 con 0x7fc5341587f0 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc532d9d700 1 --2- 172.21.15.114:0/1374760002 >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 0x7fc53415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53259c700 1 --2- 172.21.15.114:0/1374760002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc53406aff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc52400bae0 tx=0x7fc52400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e837e843bc0d12a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.826+0000 7fc53159a700 1 -- 172.21.15.114:0/1374760002 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400dcf0 con 0x7fc5341587f0 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc53159a700 1 -- 172.21.15.114:0/1374760002 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400de50 con 0x7fc5341587f0 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc53159a700 1 -- 172.21.15.114:0/1374760002 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524010690 con 0x7fc5341587f0 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 -- 172.21.15.114:0/1374760002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 msgr2=0x7fc53406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 --2- 172.21.15.114:0/1374760002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc53406aff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc52400bae0 tx=0x7fc52400bdf0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 -- 172.21.15.114:0/1374760002 shutdown_connections 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 --2- 172.21.15.114:0/1374760002 >> v2:172.21.15.172:3300/0 conn(0x7fc5340b4370 0x7fc53415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 --2- 172.21.15.114:0/1374760002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc53406aff0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 -- 172.21.15.114:0/1374760002 >> 172.21.15.114:0/1374760002 conn(0x7fc5340d0bf0 msgr2=0x7fc5340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:49.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 -- 172.21.15.114:0/1374760002 shutdown_connections 2023-11-24T06:24:49.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.827+0000 7fc538963700 1 -- 172.21.15.114:0/1374760002 wait complete. 2023-11-24T06:24:49.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.829+0000 7fc538963700 1 Processor -- start 2023-11-24T06:24:49.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.829+0000 7fc538963700 1 -- start start 2023-11-24T06:24:49.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 0x7fc53413f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 0x7fc534155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534156330 con 0x7fc5341587f0 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340b1ec0 con 0x7fc5340b4370 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534156050 con 0x7fc53415b370 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38728/0 (socket says 172.21.15.114:38728) 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 -- 172.21.15.114:0/1971335648 learned_addr learned my addr 172.21.15.114:0/1971335648 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc532d9d700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 0x7fc534155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc53259c700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 0x7fc53413f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 -- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 msgr2=0x7fc534155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 0x7fc534155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 -- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 msgr2=0x7fc53413f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 0x7fc53413f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 -- 172.21.15.114:0/1971335648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52400b740 con 0x7fc5341587f0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc53259c700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 0x7fc53413f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc532d9d700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 0x7fc534155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc531d9b700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c4b0 tx=0x7fc52800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528005d70 con 0x7fc5341587f0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5280070d0 con 0x7fc5341587f0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc534158d70 con 0x7fc5341587f0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528007440 con 0x7fc5341587f0 2023-11-24T06:24:49.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.830+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc534159330 con 0x7fc5341587f0 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.831+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534158bd0 con 0x7fc5341587f0 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.832+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5280075a0 con 0x7fc5341587f0 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.832+0000 7fc522ffd700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 0x7fc51406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.832+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc528088910 con 0x7fc5341587f0 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.832+0000 7fc53259c700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 0x7fc51406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.832+0000 7fc53259c700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 0x7fc51406d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b2c0 tx=0x7fc524013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:49.836+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc534158bd0 con 0x7fc5341587f0 2023-11-24T06:24:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.030+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc534158bd0 con 0x7fc5341587f0 2023-11-24T06:24:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.031+0000 7fc522ffd700 1 -- 172.21.15.114:0/1971335648 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc528017020 con 0x7fc5341587f0 2023-11-24T06:24:50.033 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 msgr2=0x7fc51406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 0x7fc51406d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b2c0 tx=0x7fc524013000 comp rx=0 tx=0).stop 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 msgr2=0x7fc5340b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c4b0 tx=0x7fc52800c870 comp rx=0 tx=0).stop 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 shutdown_connections 2023-11-24T06:24:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc51406aed0 0x7fc51406d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc53415b370 0x7fc534155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5340b4370 0x7fc53413f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 --2- 172.21.15.114:0/1971335648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5341587f0 0x7fc5340b1980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 >> 172.21.15.114:0/1971335648 conn(0x7fc5340d0bf0 msgr2=0x7fc5340b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 shutdown_connections 2023-11-24T06:24:50.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.033+0000 7fc538963700 1 -- 172.21.15.114:0/1971335648 wait complete. 2023-11-24T06:24:50.048 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:50.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.169+0000 7f1830ae6700 1 Processor -- start 2023-11-24T06:24:50.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.169+0000 7f1830ae6700 1 -- start start 2023-11-24T06:24:50.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f1830ae6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f182c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f1830ae6700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0ae5e0 con 0x7f182c1587f0 2023-11-24T06:24:50.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f1830ae6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c15be30 con 0x7f182c0b3c60 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f1830ae6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15c110 con 0x7f182c15b370 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182a59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f182c1587f0 0x7f182c06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54442/0 (socket says 172.21.15.114:54442) 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182a59c700 1 -- 172.21.15.114:0/538667648 learned_addr learned my addr 172.21.15.114:0/538667648 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729352560 0 0) 0x7f182c0ae5e0 con 0x7f182c1587f0 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f1829d9b700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f182c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18140035b0 con 0x7f182c1587f0 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818258029 0 0) 0x7f18140035b0 con 0x7f182c1587f0 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 msgr2=0x7f182c15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f182c15b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 >> v1:172.21.15.172:6789/0 conn(0x7f182c0b3c60 legacy=0x7f182c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0ae860 con 0x7f182c1587f0 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f181c001670 con 0x7f182c1587f0 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 >> v1:172.21.15.114:6789/0 conn(0x7f182c1587f0 legacy=0x7f182c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f1814003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1814003fb0 0x7f1814004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f1814005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15c110 con 0x7f1814004d90 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181400baf0 con 0x7f1814003fb0 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.170+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181400bcb0 con 0x7f182c15b370 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f1814005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182a59c700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f1814003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f1829d9b700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1814003fb0 0x7f1814004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 -- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 msgr2=0x7f1814003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c15b370 0x7f1814003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 -- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1814003fb0 msgr2=0x7f1814004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1814003fb0 0x7f1814004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 -- 172.21.15.114:0/538667648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140035b0 con 0x7f1814004d90 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182ad9d700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f1814005210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f182400c190 tx=0x7f1814005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf5925af30b1cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824001db0 con 0x7f1814004d90 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400ccd0 con 0x7f1814004d90 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.171+0000 7f182959a700 1 -- 172.21.15.114:0/538667648 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824003c40 con 0x7f1814004d90 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 -- 172.21.15.114:0/538667648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 msgr2=0x7f1814005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 --2- 172.21.15.114:0/538667648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f1814005210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f182400c190 tx=0x7f1814005880 comp rx=0 tx=0).stop 2023-11-24T06:24:50.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f182ad9d700 1 -- 172.21.15.114:0/538667648 reap_dead start 2023-11-24T06:24:50.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 -- 172.21.15.114:0/538667648 shutdown_connections 2023-11-24T06:24:50.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 -- 172.21.15.114:0/538667648 >> 172.21.15.114:0/538667648 conn(0x7f182c0d0bf0 msgr2=0x7f182c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 -- 172.21.15.114:0/538667648 shutdown_connections 2023-11-24T06:24:50.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.172+0000 7f1830ae6700 1 -- 172.21.15.114:0/538667648 wait complete. 2023-11-24T06:24:50.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.173+0000 7f1830ae6700 1 Processor -- start 2023-11-24T06:24:50.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.173+0000 7f1830ae6700 1 -- start start 2023-11-24T06:24:50.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c0b4390 0x7f182c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f182c09e140 0x7f182c09e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c09eeb0 con 0x7f1814004d90 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b4d50 con 0x7f182c09e140 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c146c70 con 0x7f182c0b4390 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38746/0 (socket says 172.21.15.114:38746) 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 -- 172.21.15.114:0/2029488310 learned_addr learned my addr 172.21.15.114:0/2029488310 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 -- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c0b4390 msgr2=0x7f182c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c0b4390 0x7f182c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 -- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f182c09e140 msgr2=0x7f182c09e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f182c09e140 0x7f182c09e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 -- 172.21.15.114:0/2029488310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c147100 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f182a59c700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f181c0072f0 tx=0x7f181c0076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c007890 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00f870 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.174+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c146340 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.175+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00fc20 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.175+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c146900 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.175+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c068fb0 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.176+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f181c00fd80 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.176+0000 7f181affd700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 0x7f180c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.176+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f181c089620 con 0x7f1814004d90 2023-11-24T06:24:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.176+0000 7f1829d9b700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 0x7f180c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.177+0000 7f1829d9b700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 0x7f180c06d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f182c157e30 tx=0x7f1820009670 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.181+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f181c0546d0 con 0x7f1814004d90 2023-11-24T06:24:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.369+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f182c159a10 con 0x7f1814004d90 2023-11-24T06:24:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.370+0000 7f181affd700 1 -- 172.21.15.114:0/2029488310 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f181c057cf0 con 0x7f1814004d90 2023-11-24T06:24:50.372 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 msgr2=0x7f180c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 0x7f180c06d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f182c157e30 tx=0x7f1820009670 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 msgr2=0x7f182c1439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f181c0072f0 tx=0x7f181c0076b0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 shutdown_connections 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f180c06af00 0x7f180c06d3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f182c09e140 0x7f182c09e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814004d90 0x7f182c1439e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 --2- 172.21.15.114:0/2029488310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f182c0b4390 0x7f182c0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 >> 172.21.15.114:0/2029488310 conn(0x7f182c0d0bf0 msgr2=0x7f182c159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 shutdown_connections 2023-11-24T06:24:50.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.372+0000 7f1830ae6700 1 -- 172.21.15.114:0/2029488310 wait complete. 2023-11-24T06:24:50.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:50.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.503+0000 7f70d4a67700 1 Processor -- start 2023-11-24T06:24:50.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 -- start start 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 0x7f70d00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00b1d40 con 0x7f70d00d1b30 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00b28c0 con 0x7f70d015ddd0 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.504+0000 7f70d4a67700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d006c000 con 0x7f70d015e1b0 2023-11-24T06:24:50.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70ced9d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f70d00d1b30 0x7f70d0161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48060/0 (socket says 172.21.15.114:48060) 2023-11-24T06:24:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70ce59c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 0x7f70d00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70ced9d700 1 -- 172.21.15.114:0/3042825088 learned_addr learned my addr 172.21.15.114:0/3042825088 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cd59a700 1 -- 172.21.15.114:0/3042825088 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339882930 0 0) 0x7f70d00b1d40 con 0x7f70d00d1b30 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cd59a700 1 -- 172.21.15.114:0/3042825088 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b80035b0 con 0x7f70d00d1b30 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 -- 172.21.15.114:0/3042825088 >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 msgr2=0x7f70d00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 --2- 172.21.15.114:0/3042825088 >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 0x7f70d00b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 -- 172.21.15.114:0/3042825088 >> v1:172.21.15.172:6789/0 conn(0x7f70d00d1b30 legacy=0x7f70d0161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 -- 172.21.15.114:0/3042825088 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00b1f90 con 0x7f70d015e1b0 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70ce59c700 1 --2- 172.21.15.114:0/3042825088 >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 0x7f70d00b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.505+0000 7f70cdd9b700 1 --2- 172.21.15.114:0/3042825088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d00cf000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009840 tx=0x7f70c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7947aea13d16e3d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70cd59a700 1 -- 172.21.15.114:0/3042825088 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0011830 con 0x7f70d015e1b0 2023-11-24T06:24:50.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70cd59a700 1 -- 172.21.15.114:0/3042825088 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0011990 con 0x7f70d015e1b0 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70cd59a700 1 -- 172.21.15.114:0/3042825088 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0011d00 con 0x7f70d015e1b0 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 -- 172.21.15.114:0/3042825088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 msgr2=0x7f70d00cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 --2- 172.21.15.114:0/3042825088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d00cf000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009840 tx=0x7f70c0009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 -- 172.21.15.114:0/3042825088 shutdown_connections 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 --2- 172.21.15.114:0/3042825088 >> v2:172.21.15.172:3300/0 conn(0x7f70d015ddd0 0x7f70d00b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 --2- 172.21.15.114:0/3042825088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d00cf000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 -- 172.21.15.114:0/3042825088 >> 172.21.15.114:0/3042825088 conn(0x7f70d00731a0 msgr2=0x7f70d009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 -- 172.21.15.114:0/3042825088 shutdown_connections 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.506+0000 7f70d4a67700 1 -- 172.21.15.114:0/3042825088 wait complete. 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 Processor -- start 2023-11-24T06:24:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 -- start start 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d00d1b30 0x7f70d0158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0156160 con 0x7f70d015ddd0 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0158470 con 0x7f70d00d1b30 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.508+0000 7f70d4a67700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0158750 con 0x7f70d015e1b0 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38760/0 (socket says 172.21.15.114:38760) 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 -- 172.21.15.114:0/2102277070 learned_addr learned my addr 172.21.15.114:0/2102277070 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70ced9d700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70ce59c700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d00d1b30 0x7f70d0158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 -- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 msgr2=0x7f70d01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d01556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 -- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d00d1b30 msgr2=0x7f70d0158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d00d1b30 0x7f70d0158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 -- 172.21.15.114:0/2102277070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40094a0 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70cdd9b700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70c0014040 tx=0x7f70c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0004550 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c001f070 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00038a0 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c00094a0 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.509+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d00b2ed0 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.510+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0067f60 con 0x7f70d015ddd0 2023-11-24T06:24:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.510+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70c0003a00 con 0x7f70d015ddd0 2023-11-24T06:24:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.511+0000 7f70beffd700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 0x7f70b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.511+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70c0021070 con 0x7f70d015ddd0 2023-11-24T06:24:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.511+0000 7f70ce59c700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 0x7f70b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.511+0000 7f70ce59c700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 0x7f70b006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70d009b3a0 tx=0x7f70c4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.515+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70c0055db0 con 0x7f70d015ddd0 2023-11-24T06:24:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.707+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d0158c70 con 0x7f70d015ddd0 2023-11-24T06:24:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.708+0000 7f70beffd700 1 -- 172.21.15.114:0/2102277070 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f70d0158c70 con 0x7f70d015ddd0 2023-11-24T06:24:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:50.711 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:50.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 msgr2=0x7f70b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 0x7f70b006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70d009b3a0 tx=0x7f70c4009210 comp rx=0 tx=0).stop 2023-11-24T06:24:50.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 msgr2=0x7f70d01533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70c0014040 tx=0x7f70c0004750 comp rx=0 tx=0).stop 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 shutdown_connections 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70b006aed0 0x7f70b006d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70d015e1b0 0x7f70d01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70d00d1b30 0x7f70d0158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.714 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 --2- 172.21.15.114:0/2102277070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d015ddd0 0x7f70d01533b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 >> 172.21.15.114:0/2102277070 conn(0x7f70d00731a0 msgr2=0x7f70d006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 shutdown_connections 2023-11-24T06:24:50.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.710+0000 7f70d4a67700 1 -- 172.21.15.114:0/2102277070 wait complete. 2023-11-24T06:24:50.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:50.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:50.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.846+0000 7f36e4b18700 1 Processor -- start 2023-11-24T06:24:50.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.846+0000 7f36e4b18700 1 -- start start 2023-11-24T06:24:50.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36e4b18700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f36e00746a0 0x7f36e00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36e4b18700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e00b3c60 con 0x7f36e00afc90 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36e4b18700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e00b58c0 con 0x7f36e015e160 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36e4b18700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e00b4500 con 0x7f36e00746a0 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ddd9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f36e00746a0 0x7f36e00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ded9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f36e00afc90 0x7f36e006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54478/0 (socket says 172.21.15.114:54478) 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ded9d700 1 -- 172.21.15.114:0/3820200237 learned_addr learned my addr 172.21.15.114:0/3820200237 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ddd9b700 1 -- 172.21.15.114:0/3820200237 >> v1:172.21.15.172:6789/0 conn(0x7f36e015e160 legacy=0x7f36e0073f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ddd9b700 1 -- 172.21.15.114:0/3820200237 >> v1:172.21.15.114:6789/0 conn(0x7f36e00afc90 legacy=0x7f36e006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36dd59a700 1 -- 172.21.15.114:0/3820200237 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880318922 0 0) 0x7f36e00b3c60 con 0x7f36e00afc90 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ddd9b700 1 -- 172.21.15.114:0/3820200237 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e006bdc0 con 0x7f36e00746a0 2023-11-24T06:24:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36ddd9b700 1 --2- 172.21.15.114:0/3820200237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00746a0 0x7f36e00b5da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009840 tx=0x7f36d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f93672f8e6acc83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36dd59a700 1 -- 172.21.15.114:0/3820200237 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4004230 con 0x7f36e00746a0 2023-11-24T06:24:50.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36dd59a700 1 -- 172.21.15.114:0/3820200237 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4004390 con 0x7f36e00746a0 2023-11-24T06:24:50.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.847+0000 7f36dd59a700 1 -- 172.21.15.114:0/3820200237 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40036a0 con 0x7f36e00746a0 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 -- 172.21.15.114:0/3820200237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00746a0 msgr2=0x7f36e00b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 --2- 172.21.15.114:0/3820200237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00746a0 0x7f36e00b5da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009840 tx=0x7f36d4009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 -- 172.21.15.114:0/3820200237 shutdown_connections 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 --2- 172.21.15.114:0/3820200237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00746a0 0x7f36e00b5da0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 -- 172.21.15.114:0/3820200237 >> 172.21.15.114:0/3820200237 conn(0x7f36e00d0bf0 msgr2=0x7f36e00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 -- 172.21.15.114:0/3820200237 shutdown_connections 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.848+0000 7f36e4b18700 1 -- 172.21.15.114:0/3820200237 wait complete. 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.849+0000 7f36e4b18700 1 Processor -- start 2023-11-24T06:24:50.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.849+0000 7f36e4b18700 1 -- start start 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e00746a0 0x7f36e015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e015e160 0x7f36e0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e0153450 con 0x7f36e00afc90 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01598c0 con 0x7f36e015e160 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36e4b18700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01f7b20 con 0x7f36e00746a0 2023-11-24T06:24:50.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36ddd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36ddd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38772/0 (socket says 172.21.15.114:38772) 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36ddd9b700 1 -- 172.21.15.114:0/2210839602 learned_addr learned my addr 172.21.15.114:0/2210839602 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36ddd9b700 1 -- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e00746a0 msgr2=0x7f36e015e8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.850+0000 7f36ddd9b700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e00746a0 0x7f36e015e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ddd9b700 1 -- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e015e160 msgr2=0x7f36e0154f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ddd9b700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e015e160 0x7f36e0154f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ddd9b700 1 -- 172.21.15.114:0/2210839602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40094a0 con 0x7f36e00afc90 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ddd9b700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36d40090b0 tx=0x7f36d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4011830 con 0x7f36e00afc90 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d401e070 con 0x7f36e00afc90 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e0153070 con 0x7f36e00afc90 2023-11-24T06:24:50.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4011bb0 con 0x7f36e00afc90 2023-11-24T06:24:50.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.851+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e0159570 con 0x7f36e00afc90 2023-11-24T06:24:50.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.852+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e0068fb0 con 0x7f36e00afc90 2023-11-24T06:24:50.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.852+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36d4011d10 con 0x7f36e00afc90 2023-11-24T06:24:50.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.853+0000 7f36ceffd700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 0x7f36c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:50.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.853+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36d408a8d0 con 0x7f36e00afc90 2023-11-24T06:24:50.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.853+0000 7f36de59c700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 0x7f36c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:50.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.853+0000 7f36de59c700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 0x7f36c006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006fd0 tx=0x7f36d0006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:50.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:50.857+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36d4055a10 con 0x7f36e00afc90 2023-11-24T06:24:51.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.048+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36e0153590 con 0x7f36e00afc90 2023-11-24T06:24:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.049+0000 7f36ceffd700 1 -- 172.21.15.114:0/2210839602 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36e0153590 con 0x7f36e00afc90 2023-11-24T06:24:51.052 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:51.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 msgr2=0x7f36c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 0x7f36c006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36d0006fd0 tx=0x7f36d0006bc0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 msgr2=0x7f36e0152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36d40090b0 tx=0x7f36d4004750 comp rx=0 tx=0).stop 2023-11-24T06:24:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 shutdown_connections 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36c006aed0 0x7f36c006d390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e015e160 0x7f36e0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f36e00746a0 0x7f36e015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 --2- 172.21.15.114:0/2210839602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f36e00afc90 0x7f36e0152940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 >> 172.21.15.114:0/2210839602 conn(0x7f36e00d0bf0 msgr2=0x7f36e00b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 shutdown_connections 2023-11-24T06:24:51.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.051+0000 7f36e4b18700 1 -- 172.21.15.114:0/2210839602 wait complete. 2023-11-24T06:24:51.065 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:51.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.186+0000 7f4ddae43700 1 Processor -- start 2023-11-24T06:24:51.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 -- start start 2023-11-24T06:24:51.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4dd415e080 0x7f4dd406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd406ba50 con 0x7f4dd40ce430 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd40d1200 con 0x7f4dd415e080 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.187+0000 7f4ddae43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd415e460 con 0x7f4dd40b4310 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd8bdf700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4dd415e080 0x7f4dd406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd93e0700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4dd40ce430 0x7f4dd40cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48086/0 (socket says 172.21.15.114:48086) 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd93e0700 1 -- 172.21.15.114:0/3433141352 learned_addr learned my addr 172.21.15.114:0/3433141352 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 -- 172.21.15.114:0/3433141352 >> v2:172.21.15.172:3300/0 conn(0x7f4dd415e080 msgr2=0x7f4dd406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 --2- 172.21.15.114:0/3433141352 >> v2:172.21.15.172:3300/0 conn(0x7f4dd415e080 0x7f4dd406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 -- 172.21.15.114:0/3433141352 >> v1:172.21.15.172:6789/0 conn(0x7f4dd40ce430 legacy=0x7f4dd40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 -- 172.21.15.114:0/3433141352 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd406b120 con 0x7f4dd40b4310 2023-11-24T06:24:51.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd3fff700 1 --2- 172.21.15.114:0/3433141352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd40d0900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009840 tx=0x7f4dc0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed1fb4eb44c2524b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd37fe700 1 -- 172.21.15.114:0/3433141352 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0011830 con 0x7f4dd40b4310 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd37fe700 1 -- 172.21.15.114:0/3433141352 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc0011990 con 0x7f4dd40b4310 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.188+0000 7f4dd37fe700 1 -- 172.21.15.114:0/3433141352 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0011d20 con 0x7f4dd40b4310 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 -- 172.21.15.114:0/3433141352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 msgr2=0x7f4dd40d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 --2- 172.21.15.114:0/3433141352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd40d0900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009840 tx=0x7f4dc0009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 -- 172.21.15.114:0/3433141352 shutdown_connections 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 --2- 172.21.15.114:0/3433141352 >> v2:172.21.15.172:3300/0 conn(0x7f4dd415e080 0x7f4dd406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 --2- 172.21.15.114:0/3433141352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd40d0900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 -- 172.21.15.114:0/3433141352 >> 172.21.15.114:0/3433141352 conn(0x7f4dd40b3780 msgr2=0x7f4dd40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 -- 172.21.15.114:0/3433141352 shutdown_connections 2023-11-24T06:24:51.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.189+0000 7f4ddae43700 1 -- 172.21.15.114:0/3433141352 wait complete. 2023-11-24T06:24:51.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.190+0000 7f4ddae43700 1 Processor -- start 2023-11-24T06:24:51.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.190+0000 7f4ddae43700 1 -- start start 2023-11-24T06:24:51.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd40ce430 0x7f4dd40b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd414b2a0 con 0x7f4dd415e080 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40c7080 con 0x7f4dd40ce430 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4ddae43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40c7360 con 0x7f4dd40b4310 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38788/0 (socket says 172.21.15.114:38788) 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 -- 172.21.15.114:0/2474408014 learned_addr learned my addr 172.21.15.114:0/2474408014 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd8bdf700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 -- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 msgr2=0x7f4dd414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd414ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 -- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd40ce430 msgr2=0x7f4dd40b7070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:51.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd40ce430 0x7f4dd40b7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.191+0000 7f4dd93e0700 1 -- 172.21.15.114:0/2474408014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00094a0 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4dd93e0700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00cbc0 tx=0x7f4dcc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00ab50 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc00acb0 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40c7640 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc012600 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.192+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd40c6d30 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.193+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd4068fb0 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.193+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dcc01b3b0 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.194+0000 7f4dd17fa700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 0x7f4db806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.194+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dcc089210 con 0x7f4dd415e080 2023-11-24T06:24:51.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.194+0000 7f4dd8bdf700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 0x7f4db806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.194+0000 7f4dd8bdf700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 0x7f4db806d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009a10 tx=0x7f4dc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.198+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dcc054370 con 0x7f4dd415e080 2023-11-24T06:24:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.388+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dd406fda0 con 0x7f4dd415e080 2023-11-24T06:24:51.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.388+0000 7f4dd17fa700 1 -- 172.21.15.114:0/2474408014 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4dcc057990 con 0x7f4dd415e080 2023-11-24T06:24:51.391 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:51.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 msgr2=0x7f4db806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 0x7f4db806d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009a10 tx=0x7f4dc8008040 comp rx=0 tx=0).stop 2023-11-24T06:24:51.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 msgr2=0x7f4dd414cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00cbc0 tx=0x7f4dcc00cf80 comp rx=0 tx=0).stop 2023-11-24T06:24:51.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 shutdown_connections 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db806aed0 0x7f4db806d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dd415e080 0x7f4dd414cdc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dd40b4310 0x7f4dd414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 --2- 172.21.15.114:0/2474408014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dd40ce430 0x7f4dd40b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 >> 172.21.15.114:0/2474408014 conn(0x7f4dd40b3780 msgr2=0x7f4dd40b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 shutdown_connections 2023-11-24T06:24:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.390+0000 7f4ddae43700 1 -- 172.21.15.114:0/2474408014 wait complete. 2023-11-24T06:24:51.405 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.524+0000 7fe56cd81700 1 Processor -- start 2023-11-24T06:24:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.524+0000 7fe56cd81700 1 -- start start 2023-11-24T06:24:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe56cd81700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5680746a0 0x7fe5680add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe56cd81700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56806b080 con 0x7fe56815f9e0 2023-11-24T06:24:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe56cd81700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56815c290 con 0x7fe5680cf610 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe56cd81700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680cf9f0 con 0x7fe5680746a0 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe566d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5680746a0 0x7fe5680add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe565d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe56815f9e0 0x7fe568073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54502/0 (socket says 172.21.15.114:54502) 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe566d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5680746a0 0x7fe5680add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38796/0 (socket says 172.21.15.114:38796) 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.525+0000 7fe565d9b700 1 -- 172.21.15.114:0/1020634514 learned_addr learned my addr 172.21.15.114:0/1020634514 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe566d9d700 1 -- 172.21.15.114:0/1020634514 >> v1:172.21.15.172:6789/0 conn(0x7fe5680cf610 legacy=0x7fe5680b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe566d9d700 1 -- 172.21.15.114:0/1020634514 >> v1:172.21.15.114:6789/0 conn(0x7fe56815f9e0 legacy=0x7fe568073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe566d9d700 1 -- 172.21.15.114:0/1020634514 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56815ff80 con 0x7fe5680746a0 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe56559a700 1 -- 172.21.15.114:0/1020634514 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608373631 0 0) 0x7fe56806b080 con 0x7fe56815f9e0 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe566d9d700 1 --2- 172.21.15.114:0/1020634514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5680add40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe560009d90 tx=0x7fe56000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0760607473604ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe56559a700 1 -- 172.21.15.114:0/1020634514 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000e070 con 0x7fe5680746a0 2023-11-24T06:24:51.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe56559a700 1 -- 172.21.15.114:0/1020634514 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56000bbd0 con 0x7fe5680746a0 2023-11-24T06:24:51.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.526+0000 7fe56559a700 1 -- 172.21.15.114:0/1020634514 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560012430 con 0x7fe5680746a0 2023-11-24T06:24:51.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 -- 172.21.15.114:0/1020634514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 msgr2=0x7fe5680add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 --2- 172.21.15.114:0/1020634514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5680add40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe560009d90 tx=0x7fe56000b330 comp rx=0 tx=0).stop 2023-11-24T06:24:51.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 -- 172.21.15.114:0/1020634514 shutdown_connections 2023-11-24T06:24:51.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 --2- 172.21.15.114:0/1020634514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5680add40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 -- 172.21.15.114:0/1020634514 >> 172.21.15.114:0/1020634514 conn(0x7fe5680d0bf0 msgr2=0x7fe5680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 -- 172.21.15.114:0/1020634514 shutdown_connections 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.527+0000 7fe56cd81700 1 -- 172.21.15.114:0/1020634514 wait complete. 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.528+0000 7fe56cd81700 1 Processor -- start 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.528+0000 7fe56cd81700 1 -- start start 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 0x7fe5680ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe56815f9e0 0x7fe568156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681f4b70 con 0x7fe5680746a0 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56806db60 con 0x7fe5680cf610 2023-11-24T06:24:51.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681546e0 con 0x7fe56815f9e0 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38798/0 (socket says 172.21.15.114:38798) 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 -- 172.21.15.114:0/2611053738 learned_addr learned my addr 172.21.15.114:0/2611053738 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe565d9b700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 0x7fe5680ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 -- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe56815f9e0 msgr2=0x7fe568156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe56815f9e0 0x7fe568156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 -- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 msgr2=0x7fe5680ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 0x7fe5680ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 -- 172.21.15.114:0/2611053738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009a70 con 0x7fe5680746a0 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe565d9b700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 0x7fe5680ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56659c700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c4e0 tx=0x7fe55c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0052d0 con 0x7fe5680746a0 2023-11-24T06:24:51.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c005430 con 0x7fe5680746a0 2023-11-24T06:24:51.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5681549c0 con 0x7fe5680746a0 2023-11-24T06:24:51.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0038c0 con 0x7fe5680746a0 2023-11-24T06:24:51.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.529+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681566b0 con 0x7fe5680746a0 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.530+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568156840 con 0x7fe5680746a0 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.531+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe55c003a20 con 0x7fe5680746a0 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.531+0000 7fe556ffd700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 0x7fe54806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.531+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe55c088a20 con 0x7fe5680746a0 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.531+0000 7fe565d9b700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 0x7fe54806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.536 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.532+0000 7fe565d9b700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 0x7fe54806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe56809eb10 tx=0x7fe558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.536+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe55c053b60 con 0x7fe5680746a0 2023-11-24T06:24:51.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.726+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe568068fb0 con 0x7fe5680746a0 2023-11-24T06:24:51.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.726+0000 7fe556ffd700 1 -- 172.21.15.114:0/2611053738 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe55c0170a0 con 0x7fe5680746a0 2023-11-24T06:24:51.729 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:51.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.728+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 msgr2=0x7fe54806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.728+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 0x7fe54806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe56809eb10 tx=0x7fe558009040 comp rx=0 tx=0).stop 2023-11-24T06:24:51.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.728+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 msgr2=0x7fe5681f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.728+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c4e0 tx=0x7fe55c00c8a0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 shutdown_connections 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806aed0 0x7fe54806d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe56815f9e0 0x7fe568156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5680746a0 0x7fe5681f4630 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 --2- 172.21.15.114:0/2611053738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5680cf610 0x7fe5680ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 >> 172.21.15.114:0/2611053738 conn(0x7fe5680d0bf0 msgr2=0x7fe5680b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 shutdown_connections 2023-11-24T06:24:51.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.729+0000 7fe56cd81700 1 -- 172.21.15.114:0/2611053738 wait complete. 2023-11-24T06:24:51.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:51.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.862+0000 7f932be65700 1 Processor -- start 2023-11-24T06:24:51.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.863+0000 7f932be65700 1 -- start start 2023-11-24T06:24:51.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f932be65700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 0x7f93240b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f932be65700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93240adf00 0x7f932406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f932be65700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240af950 con 0x7f932406c540 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f932be65700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932406c9f0 con 0x7f93240adf00 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f932be65700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9324144a80 con 0x7f93241446a0 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329c01700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 0x7f93240b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93240adf00 0x7f932406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93240adf00 0x7f932406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38808/0 (socket says 172.21.15.114:38808) 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329c01700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 0x7f93240b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:34464/0 (socket says 172.21.15.114:34464) 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 -- 172.21.15.114:0/446784990 learned_addr learned my addr 172.21.15.114:0/446784990 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 -- 172.21.15.114:0/446784990 >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 msgr2=0x7f93240b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:51.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 --2- 172.21.15.114:0/446784990 >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 0x7f93240b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 -- 172.21.15.114:0/446784990 >> v1:172.21.15.172:6789/0 conn(0x7f932406c540 legacy=0x7f932406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.864+0000 7f9329400700 1 -- 172.21.15.114:0/446784990 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240afc30 con 0x7f93240adf00 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.865+0000 7f9329400700 1 --2- 172.21.15.114:0/446784990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93240adf00 0x7f932406c000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f93240729a0 tx=0x7f9314009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6b3e326bd6a29c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.865+0000 7f9328bff700 1 -- 172.21.15.114:0/446784990 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314004230 con 0x7f93240adf00 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.865+0000 7f9328bff700 1 -- 172.21.15.114:0/446784990 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314004390 con 0x7f93240adf00 2023-11-24T06:24:51.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.865+0000 7f9328bff700 1 -- 172.21.15.114:0/446784990 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140036a0 con 0x7f93240adf00 2023-11-24T06:24:51.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 -- 172.21.15.114:0/446784990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93240adf00 msgr2=0x7f932406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 --2- 172.21.15.114:0/446784990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93240adf00 0x7f932406c000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f93240729a0 tx=0x7f9314009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:51.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 -- 172.21.15.114:0/446784990 shutdown_connections 2023-11-24T06:24:51.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 --2- 172.21.15.114:0/446784990 >> v2:172.21.15.172:3300/0 conn(0x7f93241446a0 0x7f93240b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 --2- 172.21.15.114:0/446784990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93240adf00 0x7f932406c000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 -- 172.21.15.114:0/446784990 >> 172.21.15.114:0/446784990 conn(0x7f9324071b90 msgr2=0x7f9324071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 -- 172.21.15.114:0/446784990 shutdown_connections 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.866+0000 7f932be65700 1 -- 172.21.15.114:0/446784990 wait complete. 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.867+0000 7f932be65700 1 Processor -- start 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.867+0000 7f932be65700 1 -- start start 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93240adf00 0x7f932406df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93241446a0 0x7f932406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932415db60 con 0x7f932406c540 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932413e5a0 con 0x7f93241446a0 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932be65700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932413cf80 con 0x7f93240adf00 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38812/0 (socket says 172.21.15.114:38812) 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 -- 172.21.15.114:0/1594249316 learned_addr learned my addr 172.21.15.114:0/1594249316 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f932a402700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93241446a0 0x7f932406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 -- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93240adf00 msgr2=0x7f932406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93240adf00 0x7f932406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 -- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93241446a0 msgr2=0x7f932406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93241446a0 0x7f932406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.868+0000 7f9329c01700 1 -- 172.21.15.114:0/1594249316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009550 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f9329c01700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f932000e870 tx=0x7f932000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320009590 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320013680 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932413c1c0 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200139f0 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.869+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932413c780 con 0x7f932406c540 2023-11-24T06:24:51.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.870+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93240753f0 con 0x7f932406c540 2023-11-24T06:24:51.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.870+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9320013b50 con 0x7f932406c540 2023-11-24T06:24:51.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.871+0000 7f931a7fc700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 0x7f930806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:51.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.871+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93200893f0 con 0x7f932406c540 2023-11-24T06:24:51.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.871+0000 7f9329400700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 0x7f930806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:51.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.871+0000 7f9329400700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 0x7f930806d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93240729a0 tx=0x7f9314010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:51.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:51.876+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9320054530 con 0x7f932406c540 2023-11-24T06:24:52.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.068+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9324067f60 con 0x7f932406c540 2023-11-24T06:24:52.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.069+0000 7f931a7fc700 1 -- 172.21.15.114:0/1594249316 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f9320057b50 con 0x7f932406c540 2023-11-24T06:24:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:52.071 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:52.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 msgr2=0x7f930806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 0x7f930806d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f93240729a0 tx=0x7f9314010000 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 msgr2=0x7f93241f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f932000e870 tx=0x7f932000eb80 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 shutdown_connections 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f930806aed0 0x7f930806d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93241446a0 0x7f932406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f932406c540 0x7f93241f3730 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 --2- 172.21.15.114:0/1594249316 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93240adf00 0x7f932406df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 >> 172.21.15.114:0/1594249316 conn(0x7f9324071b90 msgr2=0x7f93240b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 shutdown_connections 2023-11-24T06:24:52.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.071+0000 7f932be65700 1 -- 172.21.15.114:0/1594249316 wait complete. 2023-11-24T06:24:52.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:52.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:52.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.206+0000 7fd8c3ea0700 1 Processor -- start 2023-11-24T06:24:52.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.206+0000 7fd8c3ea0700 1 -- start start 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c3ea0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8bc071610 0x7fd8bc071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c3ea0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc0d2a60 con 0x7fd8bc15e090 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c3ea0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc0d2d40 con 0x7fd8bc0b6fb0 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c3ea0700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc072000 con 0x7fd8bc071610 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c1c3c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd8bc15e090 0x7fd8bc0d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54530/0 (socket says 172.21.15.114:54530) 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c1c3c700 1 -- 172.21.15.114:0/714576331 learned_addr learned my addr 172.21.15.114:0/714576331 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530169912 0 0) 0x7fd8bc0d2a60 con 0x7fd8bc15e090 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.207+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a80035b0 con 0x7fd8bc15e090 2023-11-24T06:24:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2329400381 0 0) 0x7fd8a80035b0 con 0x7fd8bc15e090 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 >> v2:172.21.15.172:3300/0 conn(0x7fd8bc071610 msgr2=0x7fd8bc071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 --2- 172.21.15.114:0/714576331 >> v2:172.21.15.172:3300/0 conn(0x7fd8bc071610 0x7fd8bc071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 >> v1:172.21.15.172:6789/0 conn(0x7fd8bc0b6fb0 legacy=0x7fd8bc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0d3020 con 0x7fd8bc15e090 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fd8ac001670 con 0x7fd8bc15e090 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 >> v1:172.21.15.114:6789/0 conn(0x7fd8bc15e090 legacy=0x7fd8bc0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 0x7fd8a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8004850 0x7fd8a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8005330 0x7fd8a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc072000 con 0x7fd8a8003a00 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a800c220 con 0x7fd8a8004850 2023-11-24T06:24:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a800c3e0 con 0x7fd8a8005330 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 0x7fd8a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 -- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8005330 msgr2=0x7fd8a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8005330 0x7fd8a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 -- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8004850 msgr2=0x7fd8a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8004850 0x7fd8a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 -- 172.21.15.114:0/714576331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80035b0 con 0x7fd8a8003a00 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c243d700 1 -- 172.21.15.114:0/714576331 reap_dead start 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 0x7fd8a8003e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001eb0 tx=0x7fd8a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1bd1ff06818667 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00ce00 con 0x7fd8a8003a00 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac005960 con 0x7fd8a8003a00 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.208+0000 7fd8c0c3a700 1 -- 172.21.15.114:0/714576331 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac005cf0 con 0x7fd8a8003a00 2023-11-24T06:24:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 msgr2=0x7fd8a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 0x7fd8a8003e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001eb0 tx=0x7fd8a8005880 comp rx=0 tx=0).stop 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/714576331 shutdown_connections 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/714576331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8a8003a00 0x7fd8a8003e00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/714576331 >> 172.21.15.114:0/714576331 conn(0x7fd8bc06b410 msgr2=0x7fd8bc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/714576331 shutdown_connections 2023-11-24T06:24:52.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.209+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/714576331 wait complete. 2023-11-24T06:24:52.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.210+0000 7fd8c3ea0700 1 Processor -- start 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.210+0000 7fd8c3ea0700 1 -- start start 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8003a00 0x7fd8bc14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8bc14c620 0x7fd8bc14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc14ce00 con 0x7fd8bc14e890 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc14d0e0 con 0x7fd8bc14c620 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c3ea0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc14d3c0 con 0x7fd8a8003a00 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38838/0 (socket says 172.21.15.114:38838) 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 -- 172.21.15.114:0/611104649 learned_addr learned my addr 172.21.15.114:0/611104649 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:52.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 -- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8003a00 msgr2=0x7fd8bc14f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8003a00 0x7fd8bc14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 -- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8bc14c620 msgr2=0x7fd8bc14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8bc14c620 0x7fd8bc14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 -- 172.21.15.114:0/611104649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc14d6a0 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.211+0000 7fd8c243d700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0d1b00 tx=0x7fd8b400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.212+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4007070 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.212+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b40071d0 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.212+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4007580 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.212+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc14d980 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.212+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc14caa0 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.213+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc068fb0 con 0x7fd8bc14e890 2023-11-24T06:24:52.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.213+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8b40076e0 con 0x7fd8bc14e890 2023-11-24T06:24:52.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.213+0000 7fd8b27fc700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 0x7fd8a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.213+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8b4088a40 con 0x7fd8bc14e890 2023-11-24T06:24:52.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.213+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 0x7fd8a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.214+0000 7fd8c1c3c700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 0x7fd8a006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.218+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8b408ae60 con 0x7fd8bc14e890 2023-11-24T06:24:52.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.406+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8bc06bad0 con 0x7fd8bc14e890 2023-11-24T06:24:52.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.407+0000 7fd8b27fc700 1 -- 172.21.15.114:0/611104649 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8b4057580 con 0x7fd8bc14e890 2023-11-24T06:24:52.409 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:52.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 msgr2=0x7fd8a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 0x7fd8a006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac00a000 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 msgr2=0x7fd8bc14b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0d1b00 tx=0x7fd8b400ca90 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 shutdown_connections 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8a006aed0 0x7fd8a006d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8bc14e890 0x7fd8bc14b9d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8a8003a00 0x7fd8bc14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 --2- 172.21.15.114:0/611104649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8bc14c620 0x7fd8bc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 >> 172.21.15.114:0/611104649 conn(0x7fd8bc06b410 msgr2=0x7fd8bc0d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 shutdown_connections 2023-11-24T06:24:52.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.409+0000 7fd8c3ea0700 1 -- 172.21.15.114:0/611104649 wait complete. 2023-11-24T06:24:52.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:52.547 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.544+0000 7f6e5f687700 1 Processor -- start 2023-11-24T06:24:52.547 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.544+0000 7f6e5f687700 1 -- start start 2023-11-24T06:24:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5f687700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6e5815e160 0x7f6e580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5f687700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 0x7f6e58073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5f687700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580b3c60 con 0x7f6e580744d0 2023-11-24T06:24:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5f687700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580b4500 con 0x7f6e5815e160 2023-11-24T06:24:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5f687700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e58074940 con 0x7f6e580afc90 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5d423700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6e5815e160 0x7f6e580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5d423700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6e5815e160 0x7f6e580b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38842/0 (socket says 172.21.15.114:38842) 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.545+0000 7f6e5d423700 1 -- 172.21.15.114:0/2609553752 learned_addr learned my addr 172.21.15.114:0/2609553752 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/2609553752 >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 0x7f6e58073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5d423700 1 -- 172.21.15.114:0/2609553752 >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 msgr2=0x7f6e58073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5d423700 1 --2- 172.21.15.114:0/2609553752 >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 0x7f6e58073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5d423700 1 -- 172.21.15.114:0/2609553752 >> v1:172.21.15.172:6789/0 conn(0x7f6e580744d0 legacy=0x7f6e5806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5d423700 1 -- 172.21.15.114:0/2609553752 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580b58c0 con 0x7f6e5815e160 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/2609553752 >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 0x7f6e58073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5d423700 1 --2- 172.21.15.114:0/2609553752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e5815e160 0x7f6e580b5da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e580d0880 tx=0x7f6e48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ab16a8a80e2093 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e4ffff700 1 -- 172.21.15.114:0/2609553752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48004230 con 0x7f6e5815e160 2023-11-24T06:24:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e4ffff700 1 -- 172.21.15.114:0/2609553752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48004390 con 0x7f6e5815e160 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e4ffff700 1 -- 172.21.15.114:0/2609553752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480036a0 con 0x7f6e5815e160 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 -- 172.21.15.114:0/2609553752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e5815e160 msgr2=0x7f6e580b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 --2- 172.21.15.114:0/2609553752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e5815e160 0x7f6e580b5da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e580d0880 tx=0x7f6e48009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 -- 172.21.15.114:0/2609553752 shutdown_connections 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 --2- 172.21.15.114:0/2609553752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e5815e160 0x7f6e580b5da0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 --2- 172.21.15.114:0/2609553752 >> v2:172.21.15.172:3300/0 conn(0x7f6e580afc90 0x7f6e58073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.546+0000 7f6e5f687700 1 -- 172.21.15.114:0/2609553752 >> 172.21.15.114:0/2609553752 conn(0x7f6e580d0bf0 msgr2=0x7f6e580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.547+0000 7f6e5f687700 1 -- 172.21.15.114:0/2609553752 shutdown_connections 2023-11-24T06:24:52.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.547+0000 7f6e5f687700 1 -- 172.21.15.114:0/2609553752 wait complete. 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 Processor -- start 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 -- start start 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 0x7f6e5815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 0x7f6e581567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.548+0000 7f6e5f687700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815f8f0 con 0x7f6e580744d0 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5f687700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815ea80 con 0x7f6e580afc90 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5f687700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815ece0 con 0x7f6e5815e160 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38848/0 (socket says 172.21.15.114:38848) 2023-11-24T06:24:52.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 -- 172.21.15.114:0/496365087 learned_addr learned my addr 172.21.15.114:0/496365087 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 0x7f6e5815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5dc24700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 0x7f6e581567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 -- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 msgr2=0x7f6e581567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 0x7f6e581567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 -- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 msgr2=0x7f6e5815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 0x7f6e5815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 -- 172.21.15.114:0/496365087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009550 con 0x7f6e580744d0 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 0x7f6e5815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5dc24700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 0x7f6e581567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5d423700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009870 tx=0x7f6e48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48011830 con 0x7f6e580744d0 2023-11-24T06:24:52.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e480119b0 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5815efc0 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48011d60 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.549+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5815f580 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.550+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580b4640 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.551+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e48024460 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.551+0000 7f6e4dffb700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 0x7f6e3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.551+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e4808af10 con 0x7f6e580744d0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.551+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 0x7f6e3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.551+0000 7f6e5cc22700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 0x7f6e3c06d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e580b2420 tx=0x7f6e54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.555+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e48056050 con 0x7f6e580744d0 2023-11-24T06:24:52.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.744+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e58068fb0 con 0x7f6e580744d0 2023-11-24T06:24:52.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.745+0000 7f6e4dffb700 1 -- 172.21.15.114:0/496365087 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e48059670 con 0x7f6e580744d0 2023-11-24T06:24:52.747 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:52.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 msgr2=0x7f6e3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 0x7f6e3c06d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e580b2420 tx=0x7f6e54008040 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 msgr2=0x7f6e581f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009870 tx=0x7f6e48004750 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 shutdown_connections 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e3c06aed0 0x7f6e3c06d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5815e160 0x7f6e581567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e580744d0 0x7f6e581f4820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 --2- 172.21.15.114:0/496365087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e580afc90 0x7f6e5815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 >> 172.21.15.114:0/496365087 conn(0x7f6e580d0bf0 msgr2=0x7f6e5806ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 shutdown_connections 2023-11-24T06:24:52.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.747+0000 7f6e5f687700 1 -- 172.21.15.114:0/496365087 wait complete. 2023-11-24T06:24:52.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:52.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.883+0000 7f06ca47d700 1 Processor -- start 2023-11-24T06:24:52.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.883+0000 7f06ca47d700 1 -- start start 2023-11-24T06:24:52.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c40b49a0 0x7f06c415b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06c406e370 0x7f06c40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06c406e880 0x7f06c406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c415e9f0 con 0x7f06c406e370 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40b52c0 con 0x7f06c406e880 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06ca47d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c415ec90 con 0x7f06c40b49a0 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06c406e370 0x7f06c40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06c406e370 0x7f06c40b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38856/0 (socket says 172.21.15.114:38856) 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 -- 172.21.15.114:0/3964521036 learned_addr learned my addr 172.21.15.114:0/3964521036 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c8a1a700 1 --2- 172.21.15.114:0/3964521036 >> v2:172.21.15.172:3300/0 conn(0x7f06c406e880 0x7f06c406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c3fff700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c40b49a0 0x7f06c415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 -- 172.21.15.114:0/3964521036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c40b49a0 msgr2=0x7f06c415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c40b49a0 0x7f06c415b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 -- 172.21.15.114:0/3964521036 >> v2:172.21.15.172:3300/0 conn(0x7f06c406e880 msgr2=0x7f06c406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 --2- 172.21.15.114:0/3964521036 >> v2:172.21.15.172:3300/0 conn(0x7f06c406e880 0x7f06c406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.884+0000 7f06c37fe700 1 -- 172.21.15.114:0/3964521036 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c415ef70 con 0x7f06c406e370 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06c37fe700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e370 0x7f06c40b4d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f06c406fc00 tx=0x7f06b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a240fa50290efc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06c2ffd700 1 -- 172.21.15.114:0/3964521036 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400b870 con 0x7f06c406e370 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06c2ffd700 1 -- 172.21.15.114:0/3964521036 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b400b9d0 con 0x7f06c406e370 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06c2ffd700 1 -- 172.21.15.114:0/3964521036 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400bd60 con 0x7f06c406e370 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 -- 172.21.15.114:0/3964521036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e370 msgr2=0x7f06c40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e370 0x7f06c40b4d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f06c406fc00 tx=0x7f06b4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 -- 172.21.15.114:0/3964521036 shutdown_connections 2023-11-24T06:24:52.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c40b49a0 0x7f06c415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 --2- 172.21.15.114:0/3964521036 >> v2:172.21.15.172:3300/0 conn(0x7f06c406e880 0x7f06c406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 --2- 172.21.15.114:0/3964521036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e370 0x7f06c40b4d80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 -- 172.21.15.114:0/3964521036 >> 172.21.15.114:0/3964521036 conn(0x7f06c40b28f0 msgr2=0x7f06c40b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:52.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 -- 172.21.15.114:0/3964521036 shutdown_connections 2023-11-24T06:24:52.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.885+0000 7f06ca47d700 1 -- 172.21.15.114:0/3964521036 wait complete. 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.886+0000 7f06ca47d700 1 Processor -- start 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 -- start start 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c406e370 0x7f06c40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c40b49a0 0x7f06c40b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40ce8c0 con 0x7f06c406e880 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40cf4b0 con 0x7f06c40b49a0 2023-11-24T06:24:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06ca47d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b07f0 con 0x7f06c406e370 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.887+0000 7f06c37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38864/0 (socket says 172.21.15.114:38864) 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 -- 172.21.15.114:0/953715496 learned_addr learned my addr 172.21.15.114:0/953715496 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c3fff700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c406e370 0x7f06c40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c8a1a700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c40b49a0 0x7f06c40b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 -- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c406e370 msgr2=0x7f06c40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c406e370 0x7f06c40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 -- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c40b49a0 msgr2=0x7f06c40b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c40b49a0 0x7f06c40b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 -- 172.21.15.114:0/953715496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b4009550 con 0x7f06c406e880 2023-11-24T06:24:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c37fe700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f06b4013000 tx=0x7f06b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004310 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b401f070 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b40036a0 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c40cd710 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.888+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c41f46f0 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.889+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c4068fb0 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.890+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06b4021030 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.890+0000 7f06c0ff9700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 0x7f06b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.890+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b408a7b0 con 0x7f06c406e880 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.890+0000 7f06c3fff700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 0x7f06b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.890+0000 7f06c3fff700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 0x7f06b006d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:52.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:52.894+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06b4055a10 con 0x7f06c406e880 2023-11-24T06:24:53.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.085+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06c40b0930 con 0x7f06c406e880 2023-11-24T06:24:53.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.085+0000 7f06c0ff9700 1 -- 172.21.15.114:0/953715496 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06b4015020 con 0x7f06c406e880 2023-11-24T06:24:53.088 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 msgr2=0x7f06b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 0x7f06b006d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 msgr2=0x7f06c4158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f06b4013000 tx=0x7f06b4004750 comp rx=0 tx=0).stop 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 shutdown_connections 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f06b006aed0 0x7f06b006d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06c40b49a0 0x7f06c40b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06c406e370 0x7f06c40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 --2- 172.21.15.114:0/953715496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06c406e880 0x7f06c4158e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 >> 172.21.15.114:0/953715496 conn(0x7f06c40b28f0 msgr2=0x7f06c4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 shutdown_connections 2023-11-24T06:24:53.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.087+0000 7f06ca47d700 1 -- 172.21.15.114:0/953715496 wait complete. 2023-11-24T06:24:53.170 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-11-24T06:24:53.171 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:24:53.171 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:53.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 Processor -- start 2023-11-24T06:24:53.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 -- start start 2023-11-24T06:24:53.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd28815e080 0x7fd2880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd28806c710 0x7fd2880afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2880ae430 con 0x7fd2880af4d0 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2880af0b0 con 0x7fd28806c710 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.290+0000 7fd28d02b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2880b05a0 con 0x7fd28815e080 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd28806c710 0x7fd2880afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd28806c710 0x7fd2880afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38874/0 (socket says 172.21.15.114:38874) 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 -- 172.21.15.114:0/2458000479 learned_addr learned my addr 172.21.15.114:0/2458000479 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd286d9d700 1 --2- 172.21.15.114:0/2458000479 >> v2:172.21.15.172:3300/0 conn(0x7fd28815e080 0x7fd2880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 -- 172.21.15.114:0/2458000479 >> v2:172.21.15.172:3300/0 conn(0x7fd28815e080 msgr2=0x7fd2880b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 --2- 172.21.15.114:0/2458000479 >> v2:172.21.15.172:3300/0 conn(0x7fd28815e080 0x7fd2880b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 -- 172.21.15.114:0/2458000479 >> v1:172.21.15.172:6789/0 conn(0x7fd2880af4d0 legacy=0x7fd28806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 -- 172.21.15.114:0/2458000479 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880ae710 con 0x7fd28806c710 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd28759e700 1 --2- 172.21.15.114:0/2458000479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd28806c710 0x7fd2880afb90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd288072a90 tx=0x7fd28000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c28bd4f7cc1b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd285d9b700 1 -- 172.21.15.114:0/2458000479 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000e070 con 0x7fd28806c710 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd285d9b700 1 -- 172.21.15.114:0/2458000479 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280003a20 con 0x7fd28806c710 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.291+0000 7fd285d9b700 1 -- 172.21.15.114:0/2458000479 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280003dd0 con 0x7fd28806c710 2023-11-24T06:24:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 -- 172.21.15.114:0/2458000479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd28806c710 msgr2=0x7fd2880afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 --2- 172.21.15.114:0/2458000479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd28806c710 0x7fd2880afb90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd288072a90 tx=0x7fd28000b330 comp rx=0 tx=0).stop 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 -- 172.21.15.114:0/2458000479 shutdown_connections 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 --2- 172.21.15.114:0/2458000479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd28806c710 0x7fd2880afb90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 --2- 172.21.15.114:0/2458000479 >> v2:172.21.15.172:3300/0 conn(0x7fd28815e080 0x7fd2880b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 -- 172.21.15.114:0/2458000479 >> 172.21.15.114:0/2458000479 conn(0x7fd288071b90 msgr2=0x7fd288071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 -- 172.21.15.114:0/2458000479 shutdown_connections 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.292+0000 7fd28d02b700 1 -- 172.21.15.114:0/2458000479 wait complete. 2023-11-24T06:24:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.293+0000 7fd28d02b700 1 Processor -- start 2023-11-24T06:24:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.293+0000 7fd28d02b700 1 -- start start 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd28806c710 0x7fd28814c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 0x7fd2880cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880ce6d0 con 0x7fd2880af4d0 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880ce9b0 con 0x7fd28806c710 2023-11-24T06:24:53.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28d02b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880cec90 con 0x7fd28815e080 2023-11-24T06:24:53.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38878/0 (socket says 172.21.15.114:38878) 2023-11-24T06:24:53.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 -- 172.21.15.114:0/766026898 learned_addr learned my addr 172.21.15.114:0/766026898 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:53.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd286d9d700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd28806c710 0x7fd28814c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28759e700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 0x7fd2880cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 -- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 msgr2=0x7fd2880cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 0x7fd2880cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 -- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd28806c710 msgr2=0x7fd28814c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd28806c710 0x7fd28814c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 -- 172.21.15.114:0/766026898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009b30 con 0x7fd2880af4d0 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28759e700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 0x7fd2880cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:53.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.294+0000 7fd28659c700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c4a0 tx=0x7fd27c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c011790 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c0118f0 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2880cef70 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c011ca0 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2881f3b30 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.295+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880cf5c0 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.296+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd27c01c460 con 0x7fd2880af4d0 2023-11-24T06:24:53.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.296+0000 7fd26f7fe700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 0x7fd27406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.296+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd27c089430 con 0x7fd2880af4d0 2023-11-24T06:24:53.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.296+0000 7fd286d9d700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 0x7fd27406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.297+0000 7fd286d9d700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 0x7fd27406d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd27800b9b0 tx=0x7fd278008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.301+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd27c0544e0 con 0x7fd2880af4d0 2023-11-24T06:24:53.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.495+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2880753f0 con 0x7fd2880af4d0 2023-11-24T06:24:53.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.496+0000 7fd26f7fe700 1 -- 172.21.15.114:0/766026898 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fd27c057b00 con 0x7fd2880af4d0 2023-11-24T06:24:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:53.499 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 msgr2=0x7fd27406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 0x7fd27406d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd27800b9b0 tx=0x7fd278008040 comp rx=0 tx=0).stop 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 msgr2=0x7fd2881f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c4a0 tx=0x7fd27c00c860 comp rx=0 tx=0).stop 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 shutdown_connections 2023-11-24T06:24:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd27406aed0 0x7fd27406d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd28815e080 0x7fd2880cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd28806c710 0x7fd28814c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 --2- 172.21.15.114:0/766026898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2880af4d0 0x7fd2881f41a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 >> 172.21.15.114:0/766026898 conn(0x7fd288071b90 msgr2=0x7fd2880b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 shutdown_connections 2023-11-24T06:24:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.498+0000 7fd28d02b700 1 -- 172.21.15.114:0/766026898 wait complete. 2023-11-24T06:24:53.513 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:53.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:53.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.633+0000 7f4778695700 1 Processor -- start 2023-11-24T06:24:53.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.634+0000 7f4778695700 1 -- start start 2023-11-24T06:24:53.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.634+0000 7f4778695700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f477006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.634+0000 7f4778695700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47700b2f20 con 0x7f477006eb20 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.634+0000 7f4778695700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477015aa00 con 0x7f477015a620 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.634+0000 7f4778695700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477006efb0 con 0x7f47700cc540 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4776c32700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f477006eb20 0x7f47700b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54596/0 (socket says 172.21.15.114:54596) 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4776c32700 1 -- 172.21.15.114:0/4269732073 learned_addr learned my addr 172.21.15.114:0/4269732073 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4775c30700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f477006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109839974 0 0) 0x7f47700b2f20 con 0x7f477006eb20 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f475c0035b0 con 0x7f477006eb20 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2315128164 0 0) 0x7f475c0035b0 con 0x7f477006eb20 2023-11-24T06:24:53.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 msgr2=0x7f477006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f477006e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 >> v1:172.21.15.172:6789/0 conn(0x7f477015a620 legacy=0x7f47700ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700ce960 con 0x7f477006eb20 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f47680019f0 con 0x7f477006eb20 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 >> v1:172.21.15.114:6789/0 conn(0x7f477006eb20 legacy=0x7f47700b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c003a30 0x7f475c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f475c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f475c004d90 0x7f475c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477006efb0 con 0x7f475c004d90 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c00cbd0 con 0x7f475c003a30 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c00cd90 con 0x7f47700cc540 2023-11-24T06:24:53.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4776c32700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f475c004d90 0x7f475c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4776431700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c003a30 0x7f475c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.635+0000 7f4775c30700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f475c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 -- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 msgr2=0x7f475c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47700cc540 0x7f475c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 -- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c003a30 msgr2=0x7f475c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c003a30 0x7f475c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 -- 172.21.15.114:0/4269732073 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0035b0 con 0x7f475c004d90 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f475c004d90 0x7f475c005210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476800a800 tx=0x7f475c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de634a81188a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476800a2f0 con 0x7f475c004d90 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768012600 con 0x7f475c004d90 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f477542f700 1 -- 172.21.15.114:0/4269732073 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680129b0 con 0x7f475c004d90 2023-11-24T06:24:53.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4778695700 1 -- 172.21.15.114:0/4269732073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f475c004d90 msgr2=0x7f475c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4778695700 1 --2- 172.21.15.114:0/4269732073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f475c004d90 0x7f475c005210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476800a800 tx=0x7f475c005880 comp rx=0 tx=0).stop 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4776c32700 1 -- 172.21.15.114:0/4269732073 reap_dead start 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4778695700 1 -- 172.21.15.114:0/4269732073 shutdown_connections 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.636+0000 7f4778695700 1 -- 172.21.15.114:0/4269732073 >> 172.21.15.114:0/4269732073 conn(0x7f47700b2050 msgr2=0x7f47700b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.637+0000 7f4778695700 1 -- 172.21.15.114:0/4269732073 shutdown_connections 2023-11-24T06:24:53.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.637+0000 7f4778695700 1 -- 172.21.15.114:0/4269732073 wait complete. 2023-11-24T06:24:53.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 Processor -- start 2023-11-24T06:24:53.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 -- start start 2023-11-24T06:24:53.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c004d90 0x7f477014adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477015ac20 0x7f477006d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477014fd80 con 0x7f477015b0a0 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770150340 con 0x7f475c004d90 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.638+0000 7f4778695700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477014b300 con 0x7f477015ac20 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38900/0 (socket says 172.21.15.114:38900) 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 -- 172.21.15.114:0/66219562 learned_addr learned my addr 172.21.15.114:0/66219562 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4775c30700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477015ac20 0x7f477006d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776431700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c004d90 0x7f477014adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 -- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477015ac20 msgr2=0x7f477006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477015ac20 0x7f477006d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 -- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c004d90 msgr2=0x7f477014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c004d90 0x7f477014adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 -- 172.21.15.114:0/66219562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700af1d0 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4776c32700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4768012000 tx=0x7f4768001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768012980 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768012b00 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477015d610 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476801e430 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.639+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477014ffd0 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.640+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770068d90 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.641+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f476801e590 con 0x7f477015b0a0 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.641+0000 7f4766ffd700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 0x7f475406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.641+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f476808b350 con 0x7f477015b0a0 2023-11-24T06:24:53.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.641+0000 7f4776431700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 0x7f475406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.642+0000 7f4776431700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 0x7f475406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4760007fd0 tx=0x7f4760009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.646+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4768056490 con 0x7f477015b0a0 2023-11-24T06:24:53.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.834+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4770150480 con 0x7f477015b0a0 2023-11-24T06:24:53.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.834+0000 7f4766ffd700 1 -- 172.21.15.114:0/66219562 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4770150480 con 0x7f477015b0a0 2023-11-24T06:24:53.837 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:53.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 msgr2=0x7f475406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 0x7f475406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4760007fd0 tx=0x7f4760009040 comp rx=0 tx=0).stop 2023-11-24T06:24:53.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 msgr2=0x7f477006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4768012000 tx=0x7f4768001d80 comp rx=0 tx=0).stop 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 shutdown_connections 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f475406aed0 0x7f475406d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477015b0a0 0x7f477006d8e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475c004d90 0x7f477014adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 --2- 172.21.15.114:0/66219562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f477015ac20 0x7f477006d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.836+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 >> 172.21.15.114:0/66219562 conn(0x7f47700b2050 msgr2=0x7f477014cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.837+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 shutdown_connections 2023-11-24T06:24:53.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.837+0000 7f4778695700 1 -- 172.21.15.114:0/66219562 wait complete. 2023-11-24T06:24:53.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.971+0000 7f7027d8a700 1 Processor -- start 2023-11-24T06:24:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.972+0000 7f7027d8a700 1 -- start start 2023-11-24T06:24:53.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7027d8a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f70201659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7027d8a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70200cd440 con 0x7f702006fab0 2023-11-24T06:24:53.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7027d8a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70200b55a0 con 0x7f7020151c30 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7027d8a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7020062900 con 0x7f70200ccfb0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7025325700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f702006fab0 0x7f70200cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54614/0 (socket says 172.21.15.114:54614) 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7025325700 1 -- 172.21.15.114:0/4246317503 learned_addr learned my addr 172.21.15.114:0/4246317503 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7026327700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f70201659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009219362 0 0) 0x7f70200cd440 con 0x7f702006fab0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c0035b0 con 0x7f702006fab0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 699139201 0 0) 0x7f700c0035b0 con 0x7f702006fab0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 msgr2=0x7f70201659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f70201659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 >> v1:172.21.15.172:6789/0 conn(0x7f7020151c30 legacy=0x7f70200ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200b52c0 con 0x7f702006fab0 2023-11-24T06:24:53.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f7010001670 con 0x7f702006fab0 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.973+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 >> v1:172.21.15.114:6789/0 conn(0x7f702006fab0 legacy=0x7f70200cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f700c003a30 0x7f700c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f700c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020062900 con 0x7f700c003a30 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00bac0 con 0x7f700c004850 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00bc80 con 0x7f70200ccfb0 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f700c003a30 0x7f700c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025325700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7026327700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f700c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 -- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 msgr2=0x7f700c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200ccfb0 0x7f700c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 -- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c004850 msgr2=0x7f700c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 -- 172.21.15.114:0/4246317503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0035b0 con 0x7f700c003a30 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7025b26700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f700c003a30 0x7f700c003e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f701c007fd0 tx=0x7f700c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f4a7db6a9f264e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c0012f0 con 0x7f700c003a30 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c0052f0 con 0x7f700c003a30 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.974+0000 7f7024b24700 1 -- 172.21.15.114:0/4246317503 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c006860 con 0x7f700c003a30 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 -- 172.21.15.114:0/4246317503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f700c003a30 msgr2=0x7f700c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 --2- 172.21.15.114:0/4246317503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f700c003a30 0x7f700c003e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f701c007fd0 tx=0x7f700c005880 comp rx=0 tx=0).stop 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7026327700 1 -- 172.21.15.114:0/4246317503 reap_dead start 2023-11-24T06:24:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 -- 172.21.15.114:0/4246317503 shutdown_connections 2023-11-24T06:24:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 -- 172.21.15.114:0/4246317503 >> 172.21.15.114:0/4246317503 conn(0x7f70200b2050 msgr2=0x7f70200b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 -- 172.21.15.114:0/4246317503 shutdown_connections 2023-11-24T06:24:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.975+0000 7f7027d8a700 1 -- 172.21.15.114:0/4246317503 wait complete. 2023-11-24T06:24:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.976+0000 7f7027d8a700 1 Processor -- start 2023-11-24T06:24:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.976+0000 7f7027d8a700 1 -- start start 2023-11-24T06:24:53.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c003a30 0x7f70200cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200c79a0 0x7f70200c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200c7e30 con 0x7f70200c6ab0 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200cbb70 con 0x7f700c003a30 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200c5a10 con 0x7f70200c79a0 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38918/0 (socket says 172.21.15.114:38918) 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 -- 172.21.15.114:0/1567465512 learned_addr learned my addr 172.21.15.114:0/1567465512 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7026327700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200c79a0 0x7f70200c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025b26700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c003a30 0x7f70200cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 -- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200c79a0 msgr2=0x7f70200c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200c79a0 0x7f70200c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 -- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c003a30 msgr2=0x7f70200cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c003a30 0x7f70200cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 -- 172.21.15.114:0/1567465512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200c5cf0 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7025325700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70100017a0 tx=0x7f7010007550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010011910 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7010011a70 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70200c5fd0 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.977+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010011e20 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.978+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70200c6590 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.978+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70200c6750 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.979+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7010010430 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.979+0000 7f70167fc700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 0x7f700406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.979+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7010089e90 con 0x7f70200c6ab0 2023-11-24T06:24:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.979+0000 7f7025b26700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 0x7f700406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.980+0000 7f7025b26700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 0x7f700406d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7020072c40 tx=0x7f701c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:53.984+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7010054f40 con 0x7f70200c6ab0 2023-11-24T06:24:54.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.176+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7020068d90 con 0x7f70200c6ab0 2023-11-24T06:24:54.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.176+0000 7f70167fc700 1 -- 172.21.15.114:0/1567465512 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7010015020 con 0x7f70200c6ab0 2023-11-24T06:24:54.179 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:54.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 msgr2=0x7f700406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 0x7f700406d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7020072c40 tx=0x7f701c011000 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 msgr2=0x7f70200c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70100017a0 tx=0x7f7010007550 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 shutdown_connections 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f700406aed0 0x7f700406d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70200c79a0 0x7f70200c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f700c003a30 0x7f70200cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 --2- 172.21.15.114:0/1567465512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70200c6ab0 0x7f70200c7460 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 >> 172.21.15.114:0/1567465512 conn(0x7f70200b2050 msgr2=0x7f70200cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:54.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 shutdown_connections 2023-11-24T06:24:54.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.178+0000 7f7027d8a700 1 -- 172.21.15.114:0/1567465512 wait complete. 2023-11-24T06:24:54.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:54.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.314+0000 7f29831dd700 1 Processor -- start 2023-11-24T06:24:54.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.314+0000 7f29831dd700 1 -- start start 2023-11-24T06:24:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f29831dd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f297c15be20 0x7f297c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f29831dd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0cdf50 0x7f297c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f29831dd700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0b4970 con 0x7f297c0b4e00 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f29831dd700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0cfed0 con 0x7f297c15be20 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f29831dd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0b4360 con 0x7f297c0cdf50 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f2980f79700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f297c15be20 0x7f297c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f2980f79700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f297c15be20 0x7f297c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38924/0 (socket says 172.21.15.114:38924) 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.315+0000 7f297bfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f297c0b4e00 0x7f297c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54632/0 (socket says 172.21.15.114:54632) 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 -- 172.21.15.114:0/641084905 learned_addr learned my addr 172.21.15.114:0/641084905 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 -- 172.21.15.114:0/641084905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0cdf50 msgr2=0x7f297c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f298177a700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0cdf50 0x7f297c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0cdf50 0x7f297c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 -- 172.21.15.114:0/641084905 >> v1:172.21.15.114:6789/0 conn(0x7f297c0b4e00 legacy=0x7f297c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f297b7fe700 1 -- 172.21.15.114:0/641084905 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885143066 0 0) 0x7f297c0b4970 con 0x7f297c0b4e00 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 -- 172.21.15.114:0/641084905 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c06f940 con 0x7f297c15be20 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f2980f79700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c15be20 0x7f297c0af790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29700098f0 tx=0x7f2970009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f363dfd1f66af47f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f297b7fe700 1 -- 172.21.15.114:0/641084905 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297000b8a0 con 0x7f297c15be20 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f297b7fe700 1 -- 172.21.15.114:0/641084905 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297000ba00 con 0x7f297c15be20 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.316+0000 7f297b7fe700 1 -- 172.21.15.114:0/641084905 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297000bd90 con 0x7f297c15be20 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 -- 172.21.15.114:0/641084905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c15be20 msgr2=0x7f297c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c15be20 0x7f297c0af790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29700098f0 tx=0x7f2970009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 -- 172.21.15.114:0/641084905 shutdown_connections 2023-11-24T06:24:54.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0cdf50 0x7f297c0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 --2- 172.21.15.114:0/641084905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c15be20 0x7f297c0af790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 -- 172.21.15.114:0/641084905 >> 172.21.15.114:0/641084905 conn(0x7f297c0b2050 msgr2=0x7f297c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:54.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 -- 172.21.15.114:0/641084905 shutdown_connections 2023-11-24T06:24:54.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.317+0000 7f29831dd700 1 -- 172.21.15.114:0/641084905 wait complete. 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.318+0000 7f29831dd700 1 Processor -- start 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.318+0000 7f29831dd700 1 -- start start 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 0x7f297c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f297c15be20 0x7f297c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c153f80 con 0x7f297c0cdf50 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15ebb0 con 0x7f297c15be20 2023-11-24T06:24:54.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29831dd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c1512c0 con 0x7f297c0b4e00 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38932/0 (socket says 172.21.15.114:38932) 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 -- 172.21.15.114:0/899517892 learned_addr learned my addr 172.21.15.114:0/899517892 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f298177a700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f297c15be20 0x7f297c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f2980f79700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 0x7f297c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 -- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 msgr2=0x7f297c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 0x7f297c150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 -- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f297c15be20 msgr2=0x7f297c1f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f297c15be20 0x7f297c1f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 -- 172.21.15.114:0/899517892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2970009550 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f2980f79700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 0x7f297c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f297bfff700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f296800c580 tx=0x7f296800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968004c80 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2968004e00 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.319+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968007310 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.320+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c15e030 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.320+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c15e5f0 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.320+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c068d90 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.321+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2968007470 con 0x7f297c0cdf50 2023-11-24T06:24:54.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.321+0000 7f29797fa700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 0x7f296006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.321+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2968089200 con 0x7f297c0cdf50 2023-11-24T06:24:54.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.321+0000 7f2980f79700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 0x7f296006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.322+0000 7f2980f79700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 0x7f296006d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f2970008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.326+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29680542b0 con 0x7f297c0cdf50 2023-11-24T06:24:54.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.514+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f297c068fb0 con 0x7f297c0cdf50 2023-11-24T06:24:54.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.515+0000 7f29797fa700 1 -- 172.21.15.114:0/899517892 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29680578d0 con 0x7f297c0cdf50 2023-11-24T06:24:54.517 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:54.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 msgr2=0x7f296006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 0x7f296006d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f2970008030 comp rx=0 tx=0).stop 2023-11-24T06:24:54.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 msgr2=0x7f297c1f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f296800c580 tx=0x7f296800c890 comp rx=0 tx=0).stop 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 shutdown_connections 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f296006aed0 0x7f296006d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f297c15be20 0x7f297c1f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f297c0b4e00 0x7f297c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 --2- 172.21.15.114:0/899517892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f297c0cdf50 0x7f297c1f4e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 >> 172.21.15.114:0/899517892 conn(0x7f297c0b2050 msgr2=0x7f297c0b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 shutdown_connections 2023-11-24T06:24:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.517+0000 7f29831dd700 1 -- 172.21.15.114:0/899517892 wait complete. 2023-11-24T06:24:54.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-24T06:24:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.649+0000 7f41eb270700 1 Processor -- start 2023-11-24T06:24:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.649+0000 7f41eb270700 1 -- start start 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41eb270700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f41e4152cc0 0x7f41e406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41eb270700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 0x7f41e406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41eb270700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e415c4b0 con 0x7f41e40cff20 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41eb270700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e415c790 con 0x7f41e4152cc0 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41eb270700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e415ca70 con 0x7f41e40b49a0 2023-11-24T06:24:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41e980d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f41e40cff20 0x7f41e40d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54648/0 (socket says 172.21.15.114:54648) 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41e980d700 1 -- 172.21.15.114:0/2002637949 learned_addr learned my addr 172.21.15.114:0/2002637949 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41e900c700 1 --2- 172.21.15.114:0/2002637949 >> v2:172.21.15.172:3300/0 conn(0x7f41e4152cc0 0x7f41e406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337686185 0 0) 0x7f41e415c4b0 con 0x7f41e40cff20 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d00035b0 con 0x7f41e40cff20 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41e880b700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 0x7f41e406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012542573 0 0) 0x7f41d00035b0 con 0x7f41e40cff20 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 msgr2=0x7f41e406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 0x7f41e406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 >> v2:172.21.15.172:3300/0 conn(0x7f41e4152cc0 msgr2=0x7f41e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 --2- 172.21.15.114:0/2002637949 >> v2:172.21.15.172:3300/0 conn(0x7f41e4152cc0 0x7f41e406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e415cd50 con 0x7f41e40cff20 2023-11-24T06:24:54.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f41dc0019f0 con 0x7f41e40cff20 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 >> v1:172.21.15.114:6789/0 conn(0x7f41e40cff20 legacy=0x7f41e40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 0x7f41d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41d0003f80 0x7f41d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41d0004d90 0x7f41d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e415c790 con 0x7f41d0003f80 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e415ca70 con 0x7f41d0004d90 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.650+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d000baf0 con 0x7f41e40b49a0 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41d0003f80 0x7f41d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 -- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 msgr2=0x7f41d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41e40b49a0 0x7f41d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 -- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41d0004d90 msgr2=0x7f41d0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41d0004d90 0x7f41d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 -- 172.21.15.114:0/2002637949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00035b0 con 0x7f41d0003f80 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41e880b700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41d0003f80 0x7f41d0004850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009b60 tx=0x7f41d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca5a9cd2dcac122 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400e6c0 con 0x7f41d0003f80 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400e820 con 0x7f41d0003f80 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.651+0000 7f41dbfff700 1 -- 172.21.15.114:0/2002637949 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400eb90 con 0x7f41d0003f80 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 -- 172.21.15.114:0/2002637949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41d0003f80 msgr2=0x7f41d0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 --2- 172.21.15.114:0/2002637949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41d0003f80 0x7f41d0004850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009b60 tx=0x7f41d0005880 comp rx=0 tx=0).stop 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41e980d700 1 -- 172.21.15.114:0/2002637949 reap_dead start 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 -- 172.21.15.114:0/2002637949 shutdown_connections 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 -- 172.21.15.114:0/2002637949 >> 172.21.15.114:0/2002637949 conn(0x7f41e4071b90 msgr2=0x7f41e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 -- 172.21.15.114:0/2002637949 shutdown_connections 2023-11-24T06:24:54.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.652+0000 7f41eb270700 1 -- 172.21.15.114:0/2002637949 wait complete. 2023-11-24T06:24:54.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.653+0000 7f41eb270700 1 Processor -- start 2023-11-24T06:24:54.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.653+0000 7f41eb270700 1 -- start start 2023-11-24T06:24:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41d0003f80 0x7f41e406da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 0x7f41e4151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e406eee0 con 0x7f41e4147510 2023-11-24T06:24:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e4151e60 con 0x7f41e4151490 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e406e260 con 0x7f41d0003f80 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38952/0 (socket says 172.21.15.114:38952) 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 -- 172.21.15.114:0/147416191 learned_addr learned my addr 172.21.15.114:0/147416191 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e980d700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 0x7f41e4151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 -- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41d0003f80 msgr2=0x7f41e406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41d0003f80 0x7f41e406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 -- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 msgr2=0x7f41e4151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 0x7f41e4151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 -- 172.21.15.114:0/147416191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e406e540 con 0x7f41e4147510 2023-11-24T06:24:54.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e980d700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 0x7f41e4151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41e880b700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f41d40090b0 tx=0x7f41d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400e6c0 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d4003ab0 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e406e820 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.654+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4003e60 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.655+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e406df70 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.655+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e40753f0 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.656+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41d4013470 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.656+0000 7f41d9ffb700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 0x7f41c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.656+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41d408ae20 con 0x7f41e4147510 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.656+0000 7f41e900c700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 0x7f41c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.657+0000 7f41e900c700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 0x7f41c806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006f10 tx=0x7f41e0006e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.661+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41d4056080 con 0x7f41e4147510 2023-11-24T06:24:54.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:54.931+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f41e4067f60 con 0x7f41e4147510 2023-11-24T06:24:55.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.021+0000 7f41d9ffb700 1 -- 172.21.15.114:0/147416191 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f41d40596a0 con 0x7f41e4147510 2023-11-24T06:24:55.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 msgr2=0x7f41c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 0x7f41c806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006f10 tx=0x7f41e0006e60 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 msgr2=0x7f41e4146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f41d40090b0 tx=0x7f41d40047c0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 shutdown_connections 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f41c806aed0 0x7f41c806d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41e4151490 0x7f41e4151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41d0003f80 0x7f41e406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 --2- 172.21.15.114:0/147416191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e4147510 0x7f41e4146c00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 >> 172.21.15.114:0/147416191 conn(0x7f41e4071b90 msgr2=0x7f41e4074700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:55.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 shutdown_connections 2023-11-24T06:24:55.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.023+0000 7f41eb270700 1 -- 172.21.15.114:0/147416191 wait complete. 2023-11-24T06:24:55.037 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:24:55.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.158+0000 7f4d53a87700 1 Processor -- start 2023-11-24T06:24:55.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 -- start start 2023-11-24T06:24:55.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c15e370 0x7f4d4c0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4d4c0c5100 0x7f4d4c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c15e750 con 0x7f4d4c0cf440 2023-11-24T06:24:55.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c099c30 con 0x7f4d4c0c5100 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.159+0000 7f4d53a87700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c0cea70 con 0x7f4d4c15e370 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d51022700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4d4c0cf440 0x7f4d4c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54666/0 (socket says 172.21.15.114:54666) 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d51022700 1 -- 172.21.15.114:0/1912877029 learned_addr learned my addr 172.21.15.114:0/1912877029 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 --2- 172.21.15.114:0/1912877029 >> v2:172.21.15.114:3300/0 conn(0x7f4d4c0c5100 0x7f4d4c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 -- 172.21.15.114:0/1912877029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c15e370 msgr2=0x7f4d4c0cef00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 --2- 172.21.15.114:0/1912877029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c15e370 0x7f4d4c0cef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 -- 172.21.15.114:0/1912877029 >> v1:172.21.15.114:6789/0 conn(0x7f4d4c0cf440 legacy=0x7f4d4c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 -- 172.21.15.114:0/1912877029 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c099e20 con 0x7f4d4c0c5100 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d50821700 1 -- 172.21.15.114:0/1912877029 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140661472 0 0) 0x7f4d4c15e750 con 0x7f4d4c0cf440 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d52024700 1 --2- 172.21.15.114:0/1912877029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c0c5100 0x7f4d4c0b0aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009d90 tx=0x7f4d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6080588995c743 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d50821700 1 -- 172.21.15.114:0/1912877029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400e070 con 0x7f4d4c0c5100 2023-11-24T06:24:55.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d50821700 1 -- 172.21.15.114:0/1912877029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4400bbd0 con 0x7f4d4c0c5100 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.160+0000 7f4d50821700 1 -- 172.21.15.114:0/1912877029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44012430 con 0x7f4d4c0c5100 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 -- 172.21.15.114:0/1912877029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c0c5100 msgr2=0x7f4d4c0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 --2- 172.21.15.114:0/1912877029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c0c5100 0x7f4d4c0b0aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009d90 tx=0x7f4d4400b330 comp rx=0 tx=0).stop 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 -- 172.21.15.114:0/1912877029 shutdown_connections 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 --2- 172.21.15.114:0/1912877029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c15e370 0x7f4d4c0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 --2- 172.21.15.114:0/1912877029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c0c5100 0x7f4d4c0b0aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 -- 172.21.15.114:0/1912877029 >> 172.21.15.114:0/1912877029 conn(0x7f4d4c0b1e90 msgr2=0x7f4d4c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 -- 172.21.15.114:0/1912877029 shutdown_connections 2023-11-24T06:24:55.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.161+0000 7f4d53a87700 1 -- 172.21.15.114:0/1912877029 wait complete. 2023-11-24T06:24:55.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.162+0000 7f4d53a87700 1 Processor -- start 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.162+0000 7f4d53a87700 1 -- start start 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c0c5100 0x7f4d4c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d4c0af0e0 0x7f4d4c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c0afc50 con 0x7f4d4c15e370 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c09a800 con 0x7f4d4c0af0e0 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d53a87700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c15d1a0 con 0x7f4d4c0c5100 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38966/0 (socket says 172.21.15.114:38966) 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 -- 172.21.15.114:0/3329044092 learned_addr learned my addr 172.21.15.114:0/3329044092 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 -- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c0c5100 msgr2=0x7f4d4c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c0c5100 0x7f4d4c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 -- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d4c0af0e0 msgr2=0x7f4d4c0afe20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d4c0af0e0 0x7f4d4c0afe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 -- 172.21.15.114:0/3329044092 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009a70 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.163+0000 7f4d51022700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c750 tx=0x7f4d4800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.164+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48004c80 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.164+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48004e00 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.164+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c0af690 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.164+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48007310 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.164+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c09a420 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.165+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4c09a280 con 0x7f4d4c15e370 2023-11-24T06:24:55.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.165+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d48007470 con 0x7f4d4c15e370 2023-11-24T06:24:55.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.165+0000 7f4d427fc700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.166+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d48089450 con 0x7f4d4c15e370 2023-11-24T06:24:55.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.166+0000 7f4d51823700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.166+0000 7f4d51823700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 0x7f4d3006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0096d0 tx=0x7f4d3c006c30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.170+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d4c09a280 con 0x7f4d4c15e370 2023-11-24T06:24:55.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.362+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4d4c09a280 con 0x7f4d4c15e370 2023-11-24T06:24:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.365+0000 7f4d427fc700 1 -- 172.21.15.114:0/3329044092 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d4c09a280 con 0x7f4d4c15e370 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 msgr2=0x7f4d3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 0x7f4d3006d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0096d0 tx=0x7f4d3c006c30 comp rx=0 tx=0).stop 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 msgr2=0x7f4d4c0aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c750 tx=0x7f4d4800cb10 comp rx=0 tx=0).stop 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 shutdown_connections 2023-11-24T06:24:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d4c0af0e0 0x7f4d4c0afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d4c0c5100 0x7f4d4c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 --2- 172.21.15.114:0/3329044092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d4c15e370 0x7f4d4c0aeba0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 >> 172.21.15.114:0/3329044092 conn(0x7f4d4c0b1e90 msgr2=0x7f4d4c160760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 shutdown_connections 2023-11-24T06:24:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.367+0000 7f4d53a87700 1 -- 172.21.15.114:0/3329044092 wait complete. 2023-11-24T06:24:55.382 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:24:55.382 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:24:55.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:24:55.482 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:24:55.482 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:55.482 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:55.500 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:55.500 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:55.557 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:55.557 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:55.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.771+0000 7fef037cb700 1 Processor -- start 2023-11-24T06:24:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.771+0000 7fef037cb700 1 -- start start 2023-11-24T06:24:55.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef037cb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feefc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef037cb700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0cfd40 con 0x7feefc0b1a70 2023-11-24T06:24:55.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef037cb700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0affa0 con 0x7feefc15ae20 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef037cb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0d0020 con 0x7feefc15b200 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef01d68700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7feefc0b1a70 0x7feefc164b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54682/0 (socket says 172.21.15.114:54682) 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.772+0000 7fef01d68700 1 -- 172.21.15.114:0/1249720648 learned_addr learned my addr 172.21.15.114:0/1249720648 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932535041 0 0) 0x7feefc0cfd40 con 0x7feefc0b1a70 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee80035b0 con 0x7feefc0b1a70 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feefc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706533322 0 0) 0x7feee80035b0 con 0x7feefc0b1a70 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 msgr2=0x7feefc0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feefc0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 >> v1:172.21.15.172:6789/0 conn(0x7feefc15ae20 legacy=0x7feefc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0b0280 con 0x7feefc0b1a70 2023-11-24T06:24:55.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7feef40019f0 con 0x7feefc0b1a70 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 >> v1:172.21.15.114:6789/0 conn(0x7feefc0b1a70 legacy=0x7feefc164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a30 0x7feee8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feee8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feee8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0d0020 con 0x7feee8004850 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee800bb70 con 0x7feee8003a30 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee800bd30 con 0x7feefc15b200 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feee8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef01567700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a30 0x7feee8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef01d68700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feee8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 -- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 msgr2=0x7feee8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feee8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 -- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a30 msgr2=0x7feee8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a30 0x7feee8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef00d66700 1 -- 172.21.15.114:0/1249720648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80035b0 con 0x7feee8004850 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.773+0000 7fef01d68700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc15b200 0x7feee8005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef00d66700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feee8004cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feef8009860 tx=0x7feee8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea249b086caa84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef800c070 con 0x7feee8004850 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef8004020 con 0x7feee8004850 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7feef3fff700 1 -- 172.21.15.114:0/1249720648 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80043b0 con 0x7feee8004850 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 -- 172.21.15.114:0/1249720648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 msgr2=0x7feee8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 --2- 172.21.15.114:0/1249720648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feee8004cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feef8009860 tx=0x7feee8005880 comp rx=0 tx=0).stop 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef01d68700 1 -- 172.21.15.114:0/1249720648 reap_dead start 2023-11-24T06:24:55.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 -- 172.21.15.114:0/1249720648 shutdown_connections 2023-11-24T06:24:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 -- 172.21.15.114:0/1249720648 >> 172.21.15.114:0/1249720648 conn(0x7feefc0b57b0 msgr2=0x7feefc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 -- 172.21.15.114:0/1249720648 shutdown_connections 2023-11-24T06:24:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.774+0000 7fef037cb700 1 -- 172.21.15.114:0/1249720648 wait complete. 2023-11-24T06:24:55.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.775+0000 7fef037cb700 1 Processor -- start 2023-11-24T06:24:55.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 -- start start 2023-11-24T06:24:55.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 0x7feefc0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 0x7feefc1f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1f2650 con 0x7feee8004850 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b1270 con 0x7feefc1f1c80 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef037cb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1508b0 con 0x7feefc0b08d0 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef01567700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef01567700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38982/0 (socket says 172.21.15.114:38982) 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef01567700 1 -- 172.21.15.114:0/2330831095 learned_addr learned my addr 172.21.15.114:0/2330831095 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef00d66700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 0x7feefc0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.776+0000 7fef01d68700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 0x7feefc1f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 -- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 msgr2=0x7feefc0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 0x7feefc0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 -- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 msgr2=0x7feefc1f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 0x7feefc1f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 -- 172.21.15.114:0/2330831095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc1f2860 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01d68700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 0x7feefc1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef00d66700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 0x7feefc0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef01567700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feeec00f7f0 tx=0x7feeec00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0052e0 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec005440 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc1f2b40 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0038c0 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.777+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc15d020 con 0x7feee8004850 2023-11-24T06:24:55.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.778+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc067f60 con 0x7feee8004850 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.778+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feeec003a20 con 0x7feee8004850 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.779+0000 7feef1ffb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 0x7feee006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.779+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeec089450 con 0x7feee8004850 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.779+0000 7fef00d66700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 0x7feee006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.779+0000 7fef00d66700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 0x7feee006d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feef8016040 tx=0x7feef8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:55.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.783+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feeec054500 con 0x7feee8004850 2023-11-24T06:24:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.975+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feefc0cde80 con 0x7feee8004850 2023-11-24T06:24:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.976+0000 7feef1ffb700 1 -- 172.21.15.114:0/2330831095 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7feeec057b20 con 0x7feee8004850 2023-11-24T06:24:55.979 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:55.979 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 msgr2=0x7feee006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 0x7feee006d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feef8016040 tx=0x7feef8008040 comp rx=0 tx=0).stop 2023-11-24T06:24:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 msgr2=0x7feefc15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feeec00f7f0 tx=0x7feeec00fbb0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 shutdown_connections 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feee006aed0 0x7feee006d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1f1c80 0x7feefc1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8004850 0x7feefc15a500 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.978+0000 7fef037cb700 1 --2- 172.21.15.114:0/2330831095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feefc0b08d0 0x7feefc0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.979+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 >> 172.21.15.114:0/2330831095 conn(0x7feefc0b57b0 msgr2=0x7feefc06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.979+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 shutdown_connections 2023-11-24T06:24:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:55.979+0000 7fef037cb700 1 -- 172.21.15.114:0/2330831095 wait complete. 2023-11-24T06:24:55.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:55.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:56.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.115+0000 7f5a9676e700 1 Processor -- start 2023-11-24T06:24:56.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.115+0000 7f5a9676e700 1 -- start start 2023-11-24T06:24:56.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a9676e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a9015f9e0 0x7f5a900add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a9676e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a90073f90 0x7f5a90074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a9676e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9006b080 con 0x7f5a900cf610 2023-11-24T06:24:56.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a9676e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a90074960 con 0x7f5a90073f90 2023-11-24T06:24:56.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a9676e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9015c290 con 0x7f5a9015f9e0 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a90073f90 0x7f5a90074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a90073f90 0x7f5a90074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38994/0 (socket says 172.21.15.114:38994) 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 -- 172.21.15.114:0/36119819 learned_addr learned my addr 172.21.15.114:0/36119819 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 -- 172.21.15.114:0/36119819 >> v2:172.21.15.172:3300/0 conn(0x7f5a9015f9e0 msgr2=0x7f5a900add40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 --2- 172.21.15.114:0/36119819 >> v2:172.21.15.172:3300/0 conn(0x7f5a9015f9e0 0x7f5a900add40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 -- 172.21.15.114:0/36119819 >> v1:172.21.15.172:6789/0 conn(0x7f5a900cf610 legacy=0x7f5a900b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 -- 172.21.15.114:0/36119819 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a900cf9f0 con 0x7f5a90073f90 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.116+0000 7f5a94d0b700 1 --2- 172.21.15.114:0/36119819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a90073f90 0x7f5a90074420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009e50 tx=0x7f5a8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a0864dcd533ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a8effd700 1 -- 172.21.15.114:0/36119819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800e070 con 0x7f5a90073f90 2023-11-24T06:24:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a8effd700 1 -- 172.21.15.114:0/36119819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a88003a20 con 0x7f5a90073f90 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a8effd700 1 -- 172.21.15.114:0/36119819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88003dd0 con 0x7f5a90073f90 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 -- 172.21.15.114:0/36119819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a90073f90 msgr2=0x7f5a90074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 --2- 172.21.15.114:0/36119819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a90073f90 0x7f5a90074420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009e50 tx=0x7f5a8800b330 comp rx=0 tx=0).stop 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 -- 172.21.15.114:0/36119819 shutdown_connections 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 --2- 172.21.15.114:0/36119819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a90073f90 0x7f5a90074420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 --2- 172.21.15.114:0/36119819 >> v2:172.21.15.172:3300/0 conn(0x7f5a9015f9e0 0x7f5a900add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 -- 172.21.15.114:0/36119819 >> 172.21.15.114:0/36119819 conn(0x7f5a900d0bf0 msgr2=0x7f5a900d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 -- 172.21.15.114:0/36119819 shutdown_connections 2023-11-24T06:24:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.117+0000 7f5a9676e700 1 -- 172.21.15.114:0/36119819 wait complete. 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.118+0000 7f5a9676e700 1 Processor -- start 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.118+0000 7f5a9676e700 1 -- start start 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a90073f90 0x7f5a90153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a9015f9e0 0x7f5a9015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9006e5e0 con 0x7f5a900cf610 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9006d9d0 con 0x7f5a9015f9e0 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a9676e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9006dcb0 con 0x7f5a90073f90 2023-11-24T06:24:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a8f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a8f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38998/0 (socket says 172.21.15.114:38998) 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.119+0000 7f5a8f7fe700 1 -- 172.21.15.114:0/4213171028 learned_addr learned my addr 172.21.15.114:0/4213171028 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 -- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a90073f90 msgr2=0x7f5a90153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a90073f90 0x7f5a90153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 -- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a9015f9e0 msgr2=0x7f5a9015a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a9015f9e0 0x7f5a9015a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 -- 172.21.15.114:0/4213171028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009b30 con 0x7f5a900cf610 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8f7fe700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b550 tx=0x7f5a8000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80011830 con 0x7f5a900cf610 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a800119b0 con 0x7f5a900cf610 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9006df90 con 0x7f5a900cf610 2023-11-24T06:24:56.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80011d60 con 0x7f5a900cf610 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.120+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a90153d90 con 0x7f5a900cf610 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.121+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a90068fb0 con 0x7f5a900cf610 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.121+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a8001a400 con 0x7f5a900cf610 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.122+0000 7f5a8cff9700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 0x7f5a7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.122+0000 7f5a8ffff700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 0x7f5a7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.122+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a80089690 con 0x7f5a900cf610 2023-11-24T06:24:56.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.122+0000 7f5a8ffff700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 0x7f5a7c06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a84003df0 tx=0x7f5a84007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.126+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a800547d0 con 0x7f5a900cf610 2023-11-24T06:24:56.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.315+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a900aecf0 con 0x7f5a900cf610 2023-11-24T06:24:56.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.316+0000 7f5a8cff9700 1 -- 172.21.15.114:0/4213171028 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a80057df0 con 0x7f5a900cf610 2023-11-24T06:24:56.318 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:56.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 msgr2=0x7f5a7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 0x7f5a7c06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a84003df0 tx=0x7f5a84007a40 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 msgr2=0x7f5a90154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b550 tx=0x7f5a8000b910 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 shutdown_connections 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a7c06aed0 0x7f5a7c06d390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a9015f9e0 0x7f5a9015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a90073f90 0x7f5a90153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 --2- 172.21.15.114:0/4213171028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a900cf610 0x7f5a90154a00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 >> 172.21.15.114:0/4213171028 conn(0x7f5a900d0bf0 msgr2=0x7f5a900b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 shutdown_connections 2023-11-24T06:24:56.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.318+0000 7f5a9676e700 1 -- 172.21.15.114:0/4213171028 wait complete. 2023-11-24T06:24:56.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:56.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.453+0000 7fb55105b700 1 Processor -- start 2023-11-24T06:24:56.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.453+0000 7fb55105b700 1 -- start start 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb55105b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb55105b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 0x7fb54c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb55105b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54c0b3c60 con 0x7fb54c15e160 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb55105b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c0b4500 con 0x7fb54c0afc90 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb55105b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0b58c0 con 0x7fb54c0746a0 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb54b59e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 0x7fb54c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb54ad9d700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb54c15e160 0x7fb54c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48308/0 (socket says 172.21.15.114:48308) 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb54ad9d700 1 -- 172.21.15.114:0/623809344 learned_addr learned my addr 172.21.15.114:0/623809344 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.454+0000 7fb54a59c700 1 --2- 172.21.15.114:0/623809344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54a59c700 1 -- 172.21.15.114:0/623809344 >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 msgr2=0x7fb54c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54a59c700 1 --2- 172.21.15.114:0/623809344 >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 0x7fb54c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54a59c700 1 -- 172.21.15.114:0/623809344 >> v1:172.21.15.172:6789/0 conn(0x7fb54c15e160 legacy=0x7fb54c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54a59c700 1 -- 172.21.15.114:0/623809344 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0b0140 con 0x7fb54c0746a0 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54b59e700 1 --2- 172.21.15.114:0/623809344 >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 0x7fb54c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb54a59c700 1 --2- 172.21.15.114:0/623809344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c0b5da0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009840 tx=0x7fb53c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2782d4d1619dbc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb549d9b700 1 -- 172.21.15.114:0/623809344 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c011830 con 0x7fb54c0746a0 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb549d9b700 1 -- 172.21.15.114:0/623809344 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c011990 con 0x7fb54c0746a0 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb549d9b700 1 -- 172.21.15.114:0/623809344 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c011d20 con 0x7fb54c0746a0 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 -- 172.21.15.114:0/623809344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 msgr2=0x7fb54c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 --2- 172.21.15.114:0/623809344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c0b5da0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009840 tx=0x7fb53c009b50 comp rx=0 tx=0).stop 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 -- 172.21.15.114:0/623809344 shutdown_connections 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 --2- 172.21.15.114:0/623809344 >> v2:172.21.15.172:3300/0 conn(0x7fb54c0afc90 0x7fb54c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 --2- 172.21.15.114:0/623809344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c0b5da0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 -- 172.21.15.114:0/623809344 >> 172.21.15.114:0/623809344 conn(0x7fb54c0d0bf0 msgr2=0x7fb54c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.455+0000 7fb55105b700 1 -- 172.21.15.114:0/623809344 shutdown_connections 2023-11-24T06:24:56.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.456+0000 7fb55105b700 1 -- 172.21.15.114:0/623809344 wait complete. 2023-11-24T06:24:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.457+0000 7fb55105b700 1 Processor -- start 2023-11-24T06:24:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.457+0000 7fb55105b700 1 -- start start 2023-11-24T06:24:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c15e160 0x7fb54c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0c6c80 con 0x7fb54c0afc90 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0c6100 con 0x7fb54c15e160 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0c63e0 con 0x7fb54c0746a0 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39012/0 (socket says 172.21.15.114:39012) 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 -- 172.21.15.114:0/452993117 learned_addr learned my addr 172.21.15.114:0/452993117 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 -- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 msgr2=0x7fb54c158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 -- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c15e160 msgr2=0x7fb54c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c15e160 0x7fb54c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 -- 172.21.15.114:0/452993117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544009a70 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb54a59c700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb53c000c00 tx=0x7fb53c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0036a0 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c01f070 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c0094a0 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.458+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003a20 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.459+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c0c69a0 con 0x7fb54c0afc90 2023-11-24T06:24:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.459+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c068fb0 con 0x7fb54c0afc90 2023-11-24T06:24:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.460+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb53c003b80 con 0x7fb54c0afc90 2023-11-24T06:24:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.460+0000 7fb5337fe700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 0x7fb53806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.460+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb53c021070 con 0x7fb54c0afc90 2023-11-24T06:24:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.460+0000 7fb54ad9d700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 0x7fb53806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.461+0000 7fb54ad9d700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 0x7fb53806d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f60 tx=0x7fb540007b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.467 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.465+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb53c055f60 con 0x7fb54c0afc90 2023-11-24T06:24:56.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.654+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb54c0ae530 con 0x7fb54c0afc90 2023-11-24T06:24:56.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.655+0000 7fb5337fe700 1 -- 172.21.15.114:0/452993117 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb53c02b020 con 0x7fb54c0afc90 2023-11-24T06:24:56.657 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:56.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.656+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 msgr2=0x7fb53806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.656+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 0x7fb53806d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f60 tx=0x7fb540007b10 comp rx=0 tx=0).stop 2023-11-24T06:24:56.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.656+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 msgr2=0x7fb54c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.656+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb53c000c00 tx=0x7fb53c004750 comp rx=0 tx=0).stop 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 shutdown_connections 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb53806aed0 0x7fb53806d390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c15e160 0x7fb54c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0746a0 0x7fb54c158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 --2- 172.21.15.114:0/452993117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb54c0afc90 0x7fb54c160690 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 >> 172.21.15.114:0/452993117 conn(0x7fb54c0d0bf0 msgr2=0x7fb54c0ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 shutdown_connections 2023-11-24T06:24:56.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.657+0000 7fb55105b700 1 -- 172.21.15.114:0/452993117 wait complete. 2023-11-24T06:24:56.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:56.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.792+0000 7f82fc908700 1 Processor -- start 2023-11-24T06:24:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.792+0000 7f82fc908700 1 -- start start 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82fc908700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82f80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82fc908700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f80b3c60 con 0x7f82f80746a0 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82fc908700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f80b58c0 con 0x7f82f815e160 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82fc908700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80b4500 con 0x7f82f80b5da0 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f5d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f82f80746a0 0x7f82f806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54728/0 (socket says 172.21.15.114:54728) 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f5d9b700 1 -- 172.21.15.114:0/1766947223 learned_addr learned my addr 172.21.15.114:0/1766947223 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402591790 0 0) 0x7f82f80b3c60 con 0x7f82f80746a0 2023-11-24T06:24:56.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e00035b0 con 0x7f82f80746a0 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f6d9d700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82f80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710606831 0 0) 0x7f82e00035b0 con 0x7f82f80746a0 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 msgr2=0x7f82f80afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82f80afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 >> v1:172.21.15.172:6789/0 conn(0x7f82f815e160 legacy=0x7f82f8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f806bdc0 con 0x7f82f80746a0 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f82e8001670 con 0x7f82f80746a0 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.793+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 >> v1:172.21.15.114:6789/0 conn(0x7f82f80746a0 legacy=0x7f82f806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82e0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82e0004060 0x7f82e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e0004d90 0x7f82e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80b4500 con 0x7f82e0004060 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e000bbd0 con 0x7f82e0004d90 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e000bd90 con 0x7f82f80b5da0 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82e0004060 0x7f82e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f659c700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82e0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f6d9d700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e0004d90 0x7f82e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 -- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 msgr2=0x7f82e0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82f80b5da0 0x7f82e0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 -- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e0004d90 msgr2=0x7f82e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82e0004d90 0x7f82e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 -- 172.21.15.114:0/1766947223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00035b0 con 0x7f82e0004060 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82e0004060 0x7f82e0004850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c240 tx=0x7f82e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c2ac91d2960835 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007c10 con 0x7f82e0004060 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800cde0 con 0x7f82e0004060 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.794+0000 7f82f559a700 1 -- 172.21.15.114:0/1766947223 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8005ba0 con 0x7f82e0004060 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 -- 172.21.15.114:0/1766947223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82e0004060 msgr2=0x7f82e0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 --2- 172.21.15.114:0/1766947223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82e0004060 0x7f82e0004850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c240 tx=0x7f82e0005880 comp rx=0 tx=0).stop 2023-11-24T06:24:56.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82f6d9d700 1 -- 172.21.15.114:0/1766947223 reap_dead start 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 -- 172.21.15.114:0/1766947223 shutdown_connections 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 -- 172.21.15.114:0/1766947223 >> 172.21.15.114:0/1766947223 conn(0x7f82f80d0bf0 msgr2=0x7f82f80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 -- 172.21.15.114:0/1766947223 shutdown_connections 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.795+0000 7f82fc908700 1 -- 172.21.15.114:0/1766947223 wait complete. 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.796+0000 7f82fc908700 1 Processor -- start 2023-11-24T06:24:56.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.796+0000 7f82fc908700 1 -- start start 2023-11-24T06:24:56.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82e0004060 0x7f82f81f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 0x7f82f81f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f806e320 con 0x7f82f8144510 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81603e0 con 0x7f82f806de70 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82fc908700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81f7e80 con 0x7f82e0004060 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39032/0 (socket says 172.21.15.114:39032) 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 -- 172.21.15.114:0/2535063660 learned_addr learned my addr 172.21.15.114:0/2535063660 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f6d9d700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 0x7f82f81f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f659c700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82e0004060 0x7f82f81f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 -- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82e0004060 msgr2=0x7f82f81f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82e0004060 0x7f82f81f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 -- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 msgr2=0x7f82f81f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 0x7f82f81f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f5d9b700 1 -- 172.21.15.114:0/2535063660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f81f7ba0 con 0x7f82f8144510 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.797+0000 7f82f6d9d700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 0x7f82f81f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82f5d9b700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f50 tx=0x7f82e8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8001640 con 0x7f82f8144510 2023-11-24T06:24:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e80070d0 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80074a0 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f81f3b20 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.798+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81f40e0 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.799+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f81f3980 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.799+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82e8019030 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.799+0000 7f82e6ffd700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 0x7f82d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.800+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82e808a4e0 con 0x7f82f8144510 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.800+0000 7f82f659c700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 0x7f82d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.800+0000 7f82f659c700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 0x7f82d806d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000c00 tx=0x7f82ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.804+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82f81f3980 con 0x7f82f8144510 2023-11-24T06:24:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.996+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82f81f3980 con 0x7f82f8144510 2023-11-24T06:24:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.997+0000 7f82e6ffd700 1 -- 172.21.15.114:0/2535063660 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82f81f3980 con 0x7f82f8144510 2023-11-24T06:24:56.999 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 msgr2=0x7f82d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 0x7f82d806d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000c00 tx=0x7f82ec009040 comp rx=0 tx=0).stop 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 msgr2=0x7f82f806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000f50 tx=0x7f82e8001a00 comp rx=0 tx=0).stop 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 shutdown_connections 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f82d806aed0 0x7f82d806d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f806de70 0x7f82f81f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82e0004060 0x7f82f81f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 --2- 172.21.15.114:0/2535063660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f8144510 0x7f82f806d930 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 >> 172.21.15.114:0/2535063660 conn(0x7f82f80d0bf0 msgr2=0x7f82f80aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 shutdown_connections 2023-11-24T06:24:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:56.999+0000 7f82fc908700 1 -- 172.21.15.114:0/2535063660 wait complete. 2023-11-24T06:24:57.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:24:57.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.132+0000 7faddba8c700 1 Processor -- start 2023-11-24T06:24:57.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 -- start start 2023-11-24T06:24:57.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fadd40cf360 0x7fadd40b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd415e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd415a790 con 0x7fadd415a3b0 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd40cf8b0 con 0x7fadd40cf360 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7faddba8c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40b4c40 con 0x7fadd415deb0 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7fadd9027700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fadd40cf360 0x7fadd40b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7fadd9828700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fadd415a3b0 0x7fadd40b3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54746/0 (socket says 172.21.15.114:54746) 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.133+0000 7fadd9828700 1 -- 172.21.15.114:0/2788040778 learned_addr learned my addr 172.21.15.114:0/2788040778 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd9027700 1 -- 172.21.15.114:0/2788040778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 msgr2=0x7fadd415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadda029700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd8826700 1 -- 172.21.15.114:0/2788040778 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694415016 0 0) 0x7fadd415a790 con 0x7fadd415a3b0 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd9027700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd9027700 1 -- 172.21.15.114:0/2788040778 >> v1:172.21.15.114:6789/0 conn(0x7fadd415a3b0 legacy=0x7fadd40b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd9027700 1 -- 172.21.15.114:0/2788040778 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4070c60 con 0x7fadd40cf360 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadda029700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd9027700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd40cf360 0x7fadd40b3a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a9e0 tx=0x7fadd000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29665ed15a5887a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd8826700 1 -- 172.21.15.114:0/2788040778 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000cb70 con 0x7fadd40cf360 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd8826700 1 -- 172.21.15.114:0/2788040778 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000ccd0 con 0x7fadd40cf360 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.134+0000 7fadd8826700 1 -- 172.21.15.114:0/2788040778 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0018620 con 0x7fadd40cf360 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 -- 172.21.15.114:0/2788040778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd40cf360 msgr2=0x7fadd40b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd40cf360 0x7fadd40b3a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a9e0 tx=0x7fadd000acf0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 -- 172.21.15.114:0/2788040778 shutdown_connections 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 --2- 172.21.15.114:0/2788040778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd40cf360 0x7fadd40b3a30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 -- 172.21.15.114:0/2788040778 >> 172.21.15.114:0/2788040778 conn(0x7fadd40b1740 msgr2=0x7fadd40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:57.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 -- 172.21.15.114:0/2788040778 shutdown_connections 2023-11-24T06:24:57.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.135+0000 7faddba8c700 1 -- 172.21.15.114:0/2788040778 wait complete. 2023-11-24T06:24:57.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.136+0000 7faddba8c700 1 Processor -- start 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.136+0000 7faddba8c700 1 -- start start 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 0x7fadd4145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd415d400 con 0x7fadd415a3b0 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd414e800 con 0x7fadd40cf360 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd414eae0 con 0x7fadd415deb0 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39046/0 (socket says 172.21.15.114:39046) 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 -- 172.21.15.114:0/1760827443 learned_addr learned my addr 172.21.15.114:0/1760827443 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadda029700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9828700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 0x7fadd4145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 -- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 msgr2=0x7fadd414e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd414e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 -- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 msgr2=0x7fadd4145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 0x7fadd4145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 -- 172.21.15.114:0/1760827443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd000a640 con 0x7fadd415a3b0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9828700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 0x7fadd4145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadd9027700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a1c0 tx=0x7fadd00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003b70 con 0x7fadd415a3b0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0003cd0 con 0x7fadd415a3b0 2023-11-24T06:24:57.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd414edc0 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd00216c0 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.137+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd414f380 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.138+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd414f540 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.139+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadd0018400 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.139+0000 7fadc67fc700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 0x7fadb806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.139+0000 7fadd9828700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 0x7fadb806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.139+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd008ac90 con 0x7fadd415a3b0 2023-11-24T06:24:57.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.140+0000 7fadd9828700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 0x7fadb806d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadc8000f60 tx=0x7fadc8007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.144+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadd0055dd0 con 0x7fadd415a3b0 2023-11-24T06:24:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.349+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fadd40751e0 con 0x7fadd415a3b0 2023-11-24T06:24:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.349+0000 7fadc67fc700 1 -- 172.21.15.114:0/1760827443 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fadd00593f0 con 0x7fadd415a3b0 2023-11-24T06:24:57.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 msgr2=0x7fadb806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 0x7fadb806d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadc8000f60 tx=0x7fadc8007ca0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 msgr2=0x7fadd414dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a1c0 tx=0x7fadd00047c0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 shutdown_connections 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadb806aed0 0x7fadb806d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadd415deb0 0x7fadd414e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd40cf360 0x7fadd4145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 --2- 172.21.15.114:0/1760827443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadd415a3b0 0x7fadd414dc90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 >> 172.21.15.114:0/1760827443 conn(0x7fadd40b1740 msgr2=0x7fadd406dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 shutdown_connections 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.351+0000 7faddba8c700 1 -- 172.21.15.114:0/1760827443 wait complete. 2023-11-24T06:24:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:24:57.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:24:57.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:57.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-24T06:24:57.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:24:57.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:57.411 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:24:57.447 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:24:57.447 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:57.468 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:24:57.468 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:57.528 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:24:57.528 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:57.588 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:57.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.706+0000 7f4fe22da700 1 Processor -- start 2023-11-24T06:24:57.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.706+0000 7f4fe22da700 1 -- start start 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fe22da700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 0x7f4fdc0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fe22da700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4fdc0cf360 0x7f4fdc0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fe22da700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc15a790 con 0x7f4fdc0b3da0 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fe22da700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc15eec0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fe22da700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc0cf8b0 con 0x7f4fdc15a3b0 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4fdc0cf360 0x7f4fdc0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4fdc0cf360 0x7f4fdc0b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39054/0 (socket says 172.21.15.114:39054) 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2204347877 learned_addr learned my addr 172.21.15.114:0/2204347877 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdbfff700 1 --2- 172.21.15.114:0/2204347877 >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 0x7f4fdc0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2204347877 >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 msgr2=0x7f4fdc0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 --2- 172.21.15.114:0/2204347877 >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 0x7f4fdc0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2204347877 >> v1:172.21.15.172:6789/0 conn(0x7f4fdc0b3da0 legacy=0x7f4fdc06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2204347877 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0b4c40 con 0x7f4fdc0cf360 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdbfff700 1 --2- 172.21.15.114:0/2204347877 >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 0x7f4fdc0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdb7fe700 1 --2- 172.21.15.114:0/2204347877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0b3860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc04fca0 tx=0x7f4fcc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929beb422a772901 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.707+0000 7f4fdaffd700 1 -- 172.21.15.114:0/2204347877 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00dc00 con 0x7f4fdc0cf360 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fdaffd700 1 -- 172.21.15.114:0/2204347877 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc00dd60 con 0x7f4fdc0cf360 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fdaffd700 1 -- 172.21.15.114:0/2204347877 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc017630 con 0x7f4fdc0cf360 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 -- 172.21.15.114:0/2204347877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 msgr2=0x7f4fdc0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2204347877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0b3860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc04fca0 tx=0x7f4fcc00bea0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 -- 172.21.15.114:0/2204347877 shutdown_connections 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2204347877 >> v2:172.21.15.172:3300/0 conn(0x7f4fdc15a3b0 0x7f4fdc0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2204347877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0b3860 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 -- 172.21.15.114:0/2204347877 >> 172.21.15.114:0/2204347877 conn(0x7f4fdc0b1740 msgr2=0x7f4fdc0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:57.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 -- 172.21.15.114:0/2204347877 shutdown_connections 2023-11-24T06:24:57.713 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.708+0000 7f4fe22da700 1 -- 172.21.15.114:0/2204347877 wait complete. 2023-11-24T06:24:57.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.709+0000 7f4fe22da700 1 Processor -- start 2023-11-24T06:24:57.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.709+0000 7f4fe22da700 1 -- start start 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fdc0b3da0 0x7f4fdc14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fdc15a3b0 0x7f4fdc15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc1585c0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc14f1f0 con 0x7f4fdc0b3da0 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe22da700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0ca700 con 0x7f4fdc15a3b0 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fdb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fdb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39060/0 (socket says 172.21.15.114:39060) 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2785051231 learned_addr learned my addr 172.21.15.114:0/2785051231 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fdbfff700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fdc0b3da0 0x7f4fdc14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.710+0000 7f4fe0877700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fdc15a3b0 0x7f4fdc15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fdc15a3b0 msgr2=0x7f4fdc15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fdc15a3b0 0x7f4fdc15ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fdc0b3da0 msgr2=0x7f4fdc14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fdc0b3da0 0x7f4fdc14ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 -- 172.21.15.114:0/2785051231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc00b7f0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fdb7fe700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc019040 tx=0x7f4fcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc009070 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc00a040 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc1582e0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004290 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.711+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc152300 con 0x7f4fdc0cf360 2023-11-24T06:24:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.712+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc0751e0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.712+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fcc015070 con 0x7f4fdc0cf360 2023-11-24T06:24:57.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.713+0000 7f4fd8ff9700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 0x7f4fc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:57.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.713+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fcc08a880 con 0x7f4fdc0cf360 2023-11-24T06:24:57.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.713+0000 7f4fdbfff700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 0x7f4fc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:57.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.713+0000 7f4fdbfff700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 0x7f4fc806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00099d0 tx=0x7f4fd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:57.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.717+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fcc0559c0 con 0x7f4fdc0cf360 2023-11-24T06:24:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.913+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fdc0b3100 con 0x7f4fdc0cf360 2023-11-24T06:24:57.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.914+0000 7f4fd8ff9700 1 -- 172.21.15.114:0/2785051231 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f4fcc027360 con 0x7f4fdc0cf360 2023-11-24T06:24:57.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:57.917 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:20.849463+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 msgr2=0x7f4fc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 0x7f4fc806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00099d0 tx=0x7f4fd0008040 comp rx=0 tx=0).stop 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 msgr2=0x7f4fdc0ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc019040 tx=0x7f4fcc004750 comp rx=0 tx=0).stop 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 shutdown_connections 2023-11-24T06:24:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4fc806aed0 0x7f4fc806d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fdc15a3b0 0x7f4fdc15ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fdc0b3da0 0x7f4fdc14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 --2- 172.21.15.114:0/2785051231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fdc0cf360 0x7f4fdc0ca1c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 >> 172.21.15.114:0/2785051231 conn(0x7f4fdc0b1740 msgr2=0x7f4fdc0ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 shutdown_connections 2023-11-24T06:24:57.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:57.916+0000 7f4fe22da700 1 -- 172.21.15.114:0/2785051231 wait complete. 2023-11-24T06:24:57.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:57.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:24:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.056+0000 7fcf12c49700 1 Processor -- start 2023-11-24T06:24:58.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.056+0000 7fcf12c49700 1 -- start start 2023-11-24T06:24:58.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf12c49700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf0c15e160 0x7fcf0c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf12c49700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c0afc90 0x7fcf0c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf12c49700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c0b3c60 con 0x7fcf0c0744d0 2023-11-24T06:24:58.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf12c49700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c0b4500 con 0x7fcf0c15e160 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf12c49700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c074960 con 0x7fcf0c0afc90 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf0c15e160 0x7fcf0c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf0c15e160 0x7fcf0c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39066/0 (socket says 172.21.15.114:39066) 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 -- 172.21.15.114:0/1517912939 learned_addr learned my addr 172.21.15.114:0/1517912939 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 -- 172.21.15.114:0/1517912939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c0afc90 msgr2=0x7fcf0c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:58.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 --2- 172.21.15.114:0/1517912939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c0afc90 0x7fcf0c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 -- 172.21.15.114:0/1517912939 >> v1:172.21.15.172:6789/0 conn(0x7fcf0c0744d0 legacy=0x7fcf0c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 -- 172.21.15.114:0/1517912939 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0b58c0 con 0x7fcf0c15e160 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.057+0000 7fcf109e5700 1 --2- 172.21.15.114:0/1517912939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c15e160 0x7fcf0c0b5da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098f0 tx=0x7fcf00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f84525fd27258a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf0b7fe700 1 -- 172.21.15.114:0/1517912939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00004230 con 0x7fcf0c15e160 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf0b7fe700 1 -- 172.21.15.114:0/1517912939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00004390 con 0x7fcf0c15e160 2023-11-24T06:24:58.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf0b7fe700 1 -- 172.21.15.114:0/1517912939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000036a0 con 0x7fcf0c15e160 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 -- 172.21.15.114:0/1517912939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c15e160 msgr2=0x7fcf0c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 --2- 172.21.15.114:0/1517912939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c15e160 0x7fcf0c0b5da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098f0 tx=0x7fcf00009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 -- 172.21.15.114:0/1517912939 shutdown_connections 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 --2- 172.21.15.114:0/1517912939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c15e160 0x7fcf0c0b5da0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 --2- 172.21.15.114:0/1517912939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c0afc90 0x7fcf0c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 -- 172.21.15.114:0/1517912939 >> 172.21.15.114:0/1517912939 conn(0x7fcf0c0d0bf0 msgr2=0x7fcf0c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 -- 172.21.15.114:0/1517912939 shutdown_connections 2023-11-24T06:24:58.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.058+0000 7fcf12c49700 1 -- 172.21.15.114:0/1517912939 wait complete. 2023-11-24T06:24:58.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.059+0000 7fcf12c49700 1 Processor -- start 2023-11-24T06:24:58.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 -- start start 2023-11-24T06:24:58.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf0c0afc90 0x7fcf0c1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c15e160 0x7fcf0c160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c1e8fa0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c145c20 con 0x7fcf0c0afc90 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf12c49700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c1e8600 con 0x7fcf0c15e160 2023-11-24T06:24:58.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf109e5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.060+0000 7fcf109e5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39072/0 (socket says 172.21.15.114:39072) 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 -- 172.21.15.114:0/2458368228 learned_addr learned my addr 172.21.15.114:0/2458368228 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf0c0afc90 0x7fcf0c1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf111e6700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c15e160 0x7fcf0c160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 -- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c15e160 msgr2=0x7fcf0c160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c15e160 0x7fcf0c160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 -- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf0c0afc90 msgr2=0x7fcf0c1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf0c0afc90 0x7fcf0c1456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 -- 172.21.15.114:0/2458368228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00009550 con 0x7fcf0c0744d0 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf109e5700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c072ed0 tx=0x7fcf00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000b860 con 0x7fcf0c0744d0 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0000b9e0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c1e88e0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000bd90 con 0x7fcf0c0744d0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.061+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c1ea360 con 0x7fcf0c0744d0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.062+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c068fb0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.063+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf0001b440 con 0x7fcf0c0744d0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.063+0000 7fcf097fa700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 0x7fcef006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.063+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcf0008afe0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.063+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 0x7fcef006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.064+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 0x7fcef006d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcef8006fd0 tx=0x7fcef8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.067+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf00056130 con 0x7fcf0c0744d0 2023-11-24T06:24:58.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.258+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf0c1ea4f0 con 0x7fcf0c0744d0 2023-11-24T06:24:58.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.259+0000 7fcf097fa700 1 -- 172.21.15.114:0/2458368228 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf00059750 con 0x7fcf0c0744d0 2023-11-24T06:24:58.262 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 msgr2=0x7fcef006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 0x7fcef006d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcef8006fd0 tx=0x7fcef8008040 comp rx=0 tx=0).stop 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 msgr2=0x7fcf0c1e9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c072ed0 tx=0x7fcf00004750 comp rx=0 tx=0).stop 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 shutdown_connections 2023-11-24T06:24:58.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef006aed0 0x7fcef006d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf0c15e160 0x7fcf0c160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf0c0744d0 0x7fcf0c1e9d80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 --2- 172.21.15.114:0/2458368228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf0c0afc90 0x7fcf0c1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 >> 172.21.15.114:0/2458368228 conn(0x7fcf0c0d0bf0 msgr2=0x7fcf0c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 shutdown_connections 2023-11-24T06:24:58.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.261+0000 7fcf12c49700 1 -- 172.21.15.114:0/2458368228 wait complete. 2023-11-24T06:24:58.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:24:58.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.396+0000 7f0b3bd54700 1 Processor -- start 2023-11-24T06:24:58.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.396+0000 7f0b3bd54700 1 -- start start 2023-11-24T06:24:58.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.396+0000 7f0b3bd54700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b3415e160 0x7f0b340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3bd54700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 0x7f0b34073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3bd54700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b340744d0 0x7f0b3406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3bd54700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b340b4500 con 0x7f0b340744d0 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3bd54700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b34074960 con 0x7f0b3415e160 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3bd54700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b3c60 con 0x7f0b340afc90 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b340744d0 0x7f0b3406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b340744d0 0x7f0b3406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39082/0 (socket says 172.21.15.114:39082) 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 -- 172.21.15.114:0/2538976788 learned_addr learned my addr 172.21.15.114:0/2538976788 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b392ef700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 0x7f0b34073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b39af0700 1 --2- 172.21.15.114:0/2538976788 >> v2:172.21.15.172:3300/0 conn(0x7f0b3415e160 0x7f0b340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 -- 172.21.15.114:0/2538976788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 msgr2=0x7f0b34073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 0x7f0b34073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 -- 172.21.15.114:0/2538976788 >> v2:172.21.15.172:3300/0 conn(0x7f0b3415e160 msgr2=0x7f0b340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 --2- 172.21.15.114:0/2538976788 >> v2:172.21.15.172:3300/0 conn(0x7f0b3415e160 0x7f0b340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 -- 172.21.15.114:0/2538976788 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b340b58c0 con 0x7f0b340744d0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b392ef700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 0x7f0b34073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b3a2f1700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3406d930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b3409efa0 tx=0x7f0b2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce39d363b3e66949 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b38aee700 1 -- 172.21.15.114:0/2538976788 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00e070 con 0x7f0b340744d0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b38aee700 1 -- 172.21.15.114:0/2538976788 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c00b910 con 0x7f0b340744d0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.397+0000 7f0b38aee700 1 -- 172.21.15.114:0/2538976788 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00bcc0 con 0x7f0b340744d0 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 -- 172.21.15.114:0/2538976788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 msgr2=0x7f0b3406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3406d930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b3409efa0 tx=0x7f0b2c00b330 comp rx=0 tx=0).stop 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 -- 172.21.15.114:0/2538976788 shutdown_connections 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3406d930 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/2538976788 >> v2:172.21.15.172:3300/0 conn(0x7f0b3415e160 0x7f0b340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/2538976788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b340afc90 0x7f0b34073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 -- 172.21.15.114:0/2538976788 >> 172.21.15.114:0/2538976788 conn(0x7f0b340d0bf0 msgr2=0x7f0b340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 -- 172.21.15.114:0/2538976788 shutdown_connections 2023-11-24T06:24:58.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.398+0000 7f0b3bd54700 1 -- 172.21.15.114:0/2538976788 wait complete. 2023-11-24T06:24:58.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.399+0000 7f0b3bd54700 1 Processor -- start 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.399+0000 7f0b3bd54700 1 -- start start 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b340afc90 0x7f0b34160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 0x7f0b340b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b341566e0 con 0x7f0b340744d0 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34153160 con 0x7f0b340afc90 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3bd54700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34160cd0 con 0x7f0b3415e160 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39084/0 (socket says 172.21.15.114:39084) 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 -- 172.21.15.114:0/3502099217 learned_addr learned my addr 172.21.15.114:0/3502099217 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3a2f1700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 0x7f0b340b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 -- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 msgr2=0x7f0b340b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 0x7f0b340b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 -- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b340afc90 msgr2=0x7f0b34160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b340afc90 0x7f0b34160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 -- 172.21.15.114:0/3502099217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c009b30 con 0x7f0b340744d0 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b3a2f1700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 0x7f0b340b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.400+0000 7f0b39af0700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400e8a0 tx=0x7f0b2400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400eea0 con 0x7f0b340744d0 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2400c850 con 0x7f0b340744d0 2023-11-24T06:24:58.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b34160fb0 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400cbc0 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b341533b0 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.401+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b340b3e70 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.402+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b2400cd20 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.402+0000 7f0b2a7fc700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 0x7f0b1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.402+0000 7f0b392ef700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 0x7f0b1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.402+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0b24089640 con 0x7f0b340744d0 2023-11-24T06:24:58.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.403+0000 7f0b392ef700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 0x7f0b1806d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.407+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b240545e0 con 0x7f0b340744d0 2023-11-24T06:24:58.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.597+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b34068fb0 con 0x7f0b340744d0 2023-11-24T06:24:58.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.598+0000 7f0b2a7fc700 1 -- 172.21.15.114:0/3502099217 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b24057c00 con 0x7f0b340744d0 2023-11-24T06:24:58.601 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:24:58.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 msgr2=0x7f0b1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 0x7f0b1806d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30008040 comp rx=0 tx=0).stop 2023-11-24T06:24:58.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 msgr2=0x7f0b3414b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400e8a0 tx=0x7f0b2400ec60 comp rx=0 tx=0).stop 2023-11-24T06:24:58.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 shutdown_connections 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0b1806aed0 0x7f0b1806d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b3415e160 0x7f0b340b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b340744d0 0x7f0b3414b790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 --2- 172.21.15.114:0/3502099217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b340afc90 0x7f0b34160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 >> 172.21.15.114:0/3502099217 conn(0x7f0b340d0bf0 msgr2=0x7f0b3406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 shutdown_connections 2023-11-24T06:24:58.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.600+0000 7f0b3bd54700 1 -- 172.21.15.114:0/3502099217 wait complete. 2023-11-24T06:24:58.615 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:24:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.735+0000 7f38dc0a6700 1 Processor -- start 2023-11-24T06:24:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 -- start start 2023-11-24T06:24:58.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38d40aedc0 0x7f38d40ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d406e400 con 0x7f38d40ad870 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d406fde0 con 0x7f38d40aedc0 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.736+0000 7f38dc0a6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40af6e0 con 0x7f38d415e290 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:33638/0 (socket says 172.21.15.114:33638) 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9e42700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38d40aedc0 0x7f38d40ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 -- 172.21.15.114:0/3194605192 learned_addr learned my addr 172.21.15.114:0/3194605192 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 -- 172.21.15.114:0/3194605192 >> v2:172.21.15.172:3300/0 conn(0x7f38d40aedc0 msgr2=0x7f38d40ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d8e40700 1 -- 172.21.15.114:0/3194605192 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958172799 0 0) 0x7f38d406e400 con 0x7f38d40ad870 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 --2- 172.21.15.114:0/3194605192 >> v2:172.21.15.172:3300/0 conn(0x7f38d40aedc0 0x7f38d40ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 -- 172.21.15.114:0/3194605192 >> v1:172.21.15.172:6789/0 conn(0x7f38d40ad870 legacy=0x7f38d406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 -- 172.21.15.114:0/3194605192 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d406e630 con 0x7f38d415e290 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d9641700 1 --2- 172.21.15.114:0/3194605192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098f0 tx=0x7f38d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae79b14f34eb6dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d8e40700 1 -- 172.21.15.114:0/3194605192 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011830 con 0x7f38d415e290 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d8e40700 1 -- 172.21.15.114:0/3194605192 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0011990 con 0x7f38d415e290 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.737+0000 7f38d8e40700 1 -- 172.21.15.114:0/3194605192 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011d20 con 0x7f38d415e290 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 -- 172.21.15.114:0/3194605192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 msgr2=0x7f38d40af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/3194605192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38d00098f0 tx=0x7f38d0009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 -- 172.21.15.114:0/3194605192 shutdown_connections 2023-11-24T06:24:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/3194605192 >> v2:172.21.15.172:3300/0 conn(0x7f38d40aedc0 0x7f38d40ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/3194605192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d40af1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 -- 172.21.15.114:0/3194605192 >> 172.21.15.114:0/3194605192 conn(0x7f38d40b2050 msgr2=0x7f38d40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 -- 172.21.15.114:0/3194605192 shutdown_connections 2023-11-24T06:24:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.738+0000 7f38dc0a6700 1 -- 172.21.15.114:0/3194605192 wait complete. 2023-11-24T06:24:58.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.739+0000 7f38dc0a6700 1 Processor -- start 2023-11-24T06:24:58.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.739+0000 7f38dc0a6700 1 -- start start 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d40aedc0 0x7f38d415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41f5320 con 0x7f38d40ad870 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d414f540 con 0x7f38d40aedc0 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38dc0a6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41f4910 con 0x7f38d415e290 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39098/0 (socket says 172.21.15.114:39098) 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 -- 172.21.15.114:0/157433992 learned_addr learned my addr 172.21.15.114:0/157433992 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9641700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d40aedc0 0x7f38d415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38da643700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 -- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 msgr2=0x7f38d415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 -- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d40aedc0 msgr2=0x7f38d415f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d40aedc0 0x7f38d415f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.740+0000 7f38d9e42700 1 -- 172.21.15.114:0/157433992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009550 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38d9e42700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38c400e9b0 tx=0x7f38c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4009710 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c400ee50 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400ca50 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41f4630 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.741+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d41f4f40 con 0x7f38d40ad870 2023-11-24T06:24:58.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.742+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d4068d90 con 0x7f38d40ad870 2023-11-24T06:24:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.742+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38c400cbb0 con 0x7f38d40ad870 2023-11-24T06:24:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.743+0000 7f38ca7fc700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 0x7f38b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.743+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f38c4089690 con 0x7f38d40ad870 2023-11-24T06:24:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.743+0000 7f38d9641700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 0x7f38b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.743+0000 7f38d9641700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 0x7f38b806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38d415e670 tx=0x7f38d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:58.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.747+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38c4054630 con 0x7f38d40ad870 2023-11-24T06:24:58.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.941+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38d41f5590 con 0x7f38d40ad870 2023-11-24T06:24:58.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.941+0000 7f38ca7fc700 1 -- 172.21.15.114:0/157433992 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38c400ce60 con 0x7f38d40ad870 2023-11-24T06:24:58.944 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:24:58.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.943+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 msgr2=0x7f38b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.943+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 0x7f38b806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f38d415e670 tx=0x7f38d0008030 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.943+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 msgr2=0x7f38d414f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.943+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38c400e9b0 tx=0x7f38c400ecc0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 shutdown_connections 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38b806aed0 0x7f38b806d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38d415e290 0x7f38d415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38d40ad870 0x7f38d414f000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 --2- 172.21.15.114:0/157433992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d40aedc0 0x7f38d415f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 >> 172.21.15.114:0/157433992 conn(0x7f38d40b2050 msgr2=0x7f38d4159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:58.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 shutdown_connections 2023-11-24T06:24:58.948 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:58.944+0000 7f38dc0a6700 1 -- 172.21.15.114:0/157433992 wait complete. 2023-11-24T06:24:58.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:24:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.079+0000 7fd96ee60700 1 Processor -- start 2023-11-24T06:24:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.079+0000 7fd96ee60700 1 -- start start 2023-11-24T06:24:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96ee60700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd9680d1570 0x7fd9680d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96ee60700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680b2fc0 0x7fd9680d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96ee60700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9680d2d10 con 0x7fd96815dfc0 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96ee60700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9680cefd0 con 0x7fd9680d1570 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96ee60700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680cf2b0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd9680d1570 0x7fd9680d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd9680d1570 0x7fd9680d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39108/0 (socket says 172.21.15.114:39108) 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 -- 172.21.15.114:0/931442515 learned_addr learned my addr 172.21.15.114:0/931442515 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd96d3fd700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680b2fc0 0x7fd9680d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 -- 172.21.15.114:0/931442515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680b2fc0 msgr2=0x7fd9680d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680b2fc0 0x7fd9680d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 -- 172.21.15.114:0/931442515 >> v1:172.21.15.172:6789/0 conn(0x7fd96815dfc0 legacy=0x7fd9680d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 -- 172.21.15.114:0/931442515 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680d2fb0 con 0x7fd9680d1570 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd967fff700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680d1570 0x7fd9680d34d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098f0 tx=0x7fd954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4102a80c26c400c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd9677fe700 1 -- 172.21.15.114:0/931442515 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400b8a0 con 0x7fd9680d1570 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd9677fe700 1 -- 172.21.15.114:0/931442515 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95400ba00 con 0x7fd9680d1570 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.080+0000 7fd9677fe700 1 -- 172.21.15.114:0/931442515 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400bd90 con 0x7fd9680d1570 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 -- 172.21.15.114:0/931442515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680d1570 msgr2=0x7fd9680d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680d1570 0x7fd9680d34d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098f0 tx=0x7fd954009c00 comp rx=0 tx=0).stop 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 -- 172.21.15.114:0/931442515 shutdown_connections 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680b2fc0 0x7fd9680d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 --2- 172.21.15.114:0/931442515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680d1570 0x7fd9680d34d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 -- 172.21.15.114:0/931442515 >> 172.21.15.114:0/931442515 conn(0x7fd96806b410 msgr2=0x7fd96806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 -- 172.21.15.114:0/931442515 shutdown_connections 2023-11-24T06:24:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.081+0000 7fd96ee60700 1 -- 172.21.15.114:0/931442515 wait complete. 2023-11-24T06:24:59.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.082+0000 7fd96ee60700 1 Processor -- start 2023-11-24T06:24:59.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.082+0000 7fd96ee60700 1 -- start start 2023-11-24T06:24:59.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680d1570 0x7fd9680b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 0x7fd9680b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96815c310 con 0x7fd9680b2fc0 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd968145650 con 0x7fd96815dfc0 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96ee60700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9681484a0 con 0x7fd9680d1570 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96cbfc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96cbfc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39112/0 (socket says 172.21.15.114:39112) 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96cbfc700 1 -- 172.21.15.114:0/3025391410 learned_addr learned my addr 172.21.15.114:0/3025391410 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd967fff700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680d1570 0x7fd9680b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.083+0000 7fd96d3fd700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 0x7fd9680b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 -- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680d1570 msgr2=0x7fd9680b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680d1570 0x7fd9680b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 -- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 msgr2=0x7fd9680b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 0x7fd9680b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 -- 172.21.15.114:0/3025391410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954009550 con 0x7fd9680b2fc0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96d3fd700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 0x7fd9680b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96cbfc700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c4e0 tx=0x7fd95c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0052d0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c005430 con 0x7fd9680b2fc0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9681458a0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0038c0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.084+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd968145e60 con 0x7fd9680b2fc0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.085+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd968145ff0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.085+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd95c003a20 con 0x7fd9680b2fc0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.086+0000 7fd9657fa700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 0x7fd94c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.086+0000 7fd967fff700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 0x7fd94c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.086+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd95c088aa0 con 0x7fd9680b2fc0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.086+0000 7fd967fff700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 0x7fd94c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd954009c40 tx=0x7fd954008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.092 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.090+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd95c053b60 con 0x7fd9680b2fc0 2023-11-24T06:24:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.298+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd968067f60 con 0x7fd9680b2fc0 2023-11-24T06:24:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.299+0000 7fd9657fa700 1 -- 172.21.15.114:0/3025391410 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd95c017080 con 0x7fd9680b2fc0 2023-11-24T06:24:59.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 msgr2=0x7fd94c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 0x7fd94c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd954009c40 tx=0x7fd954008030 comp rx=0 tx=0).stop 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 msgr2=0x7fd96815bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c4e0 tx=0x7fd95c00c8a0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 shutdown_connections 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd94c06aed0 0x7fd94c06d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd96815dfc0 0x7fd9680b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9680b2fc0 0x7fd96815bdd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 --2- 172.21.15.114:0/3025391410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9680d1570 0x7fd9680b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 >> 172.21.15.114:0/3025391410 conn(0x7fd96806b410 msgr2=0x7fd968160530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 shutdown_connections 2023-11-24T06:24:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.301+0000 7fd96ee60700 1 -- 172.21.15.114:0/3025391410 wait complete. 2023-11-24T06:24:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:24:59.316 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:24:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:59.316 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-24T06:24:59.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:24:59.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:24:59.362 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:24:59.469 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:59.469 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:59.487 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:59.487 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:59.544 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:24:59.545 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:24:59.602 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:24:59.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.760+0000 7ff3ff41b700 1 Processor -- start 2023-11-24T06:24:59.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 -- start start 2023-11-24T06:24:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 0x7ff3f80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3f80af4d0 0x7ff3f80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f806c000 0x7ff3f806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f80af0b0 con 0x7ff3f80af4d0 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f806c9f0 con 0x7ff3f815e080 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.761+0000 7ff3ff41b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80b05a0 con 0x7ff3f806c000 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3f80af4d0 0x7ff3f80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3f80af4d0 0x7ff3f80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39122/0 (socket says 172.21.15.114:39122) 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 -- 172.21.15.114:0/283182829 learned_addr learned my addr 172.21.15.114:0/283182829 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fd9b8700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f806c000 0x7ff3f806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fd1b7700 1 --2- 172.21.15.114:0/283182829 >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 0x7ff3f80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 -- 172.21.15.114:0/283182829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f806c000 msgr2=0x7ff3f806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f806c000 0x7ff3f806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 -- 172.21.15.114:0/283182829 >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 msgr2=0x7ff3f80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/283182829 >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 0x7ff3f80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 -- 172.21.15.114:0/283182829 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80ae430 con 0x7ff3f80af4d0 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fd1b7700 1 --2- 172.21.15.114:0/283182829 >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 0x7ff3f80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f80af4d0 0x7ff3f80afb90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8072a90 tx=0x7ff3f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4faef1318e39181 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3ebfff700 1 -- 172.21.15.114:0/283182829 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400b870 con 0x7ff3f80af4d0 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3ebfff700 1 -- 172.21.15.114:0/283182829 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f400b9d0 con 0x7ff3f80af4d0 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.762+0000 7ff3ebfff700 1 -- 172.21.15.114:0/283182829 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400bd60 con 0x7ff3f80af4d0 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 -- 172.21.15.114:0/283182829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f80af4d0 msgr2=0x7ff3f80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f80af4d0 0x7ff3f80afb90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8072a90 tx=0x7ff3f4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 -- 172.21.15.114:0/283182829 shutdown_connections 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f806c000 0x7ff3f806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/283182829 >> v2:172.21.15.172:3300/0 conn(0x7ff3f815e080 0x7ff3f80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/283182829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f80af4d0 0x7ff3f80afb90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 -- 172.21.15.114:0/283182829 >> 172.21.15.114:0/283182829 conn(0x7ff3f8071b90 msgr2=0x7ff3f8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 -- 172.21.15.114:0/283182829 shutdown_connections 2023-11-24T06:24:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.763+0000 7ff3ff41b700 1 -- 172.21.15.114:0/283182829 wait complete. 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.764+0000 7ff3ff41b700 1 Processor -- start 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.764+0000 7ff3ff41b700 1 -- start start 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 0x7ff3f813f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f815e080 0x7ff3f8158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8161390 con 0x7ff3f806c000 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8154fc0 con 0x7ff3f815e080 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3ff41b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f81552a0 con 0x7ff3f80af4d0 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39126/0 (socket says 172.21.15.114:39126) 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 -- 172.21.15.114:0/934128456 learned_addr learned my addr 172.21.15.114:0/934128456 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 0x7ff3f813f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 -- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 msgr2=0x7ff3f813f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 0x7ff3f813f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 -- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f815e080 msgr2=0x7ff3f8158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd9b8700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f815e080 0x7ff3f8158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f815e080 0x7ff3f8158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 -- 172.21.15.114:0/934128456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4009550 con 0x7ff3f806c000 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 0x7ff3f813f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.765+0000 7ff3fd1b7700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c7f0 tx=0x7ff3ec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.766+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011800 con 0x7ff3f806c000 2023-11-24T06:24:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.766+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec011960 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.766+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f8155580 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.766+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011d10 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.766+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f813f960 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.767+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f8158480 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.767+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3ec018360 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.767+0000 7ff3e9ffb700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 0x7ff3dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.768+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff3ec089700 con 0x7ff3f806c000 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.768+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 0x7ff3dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:24:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.768+0000 7ff3fc9b6700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 0x7ff3dc06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8072a90 tx=0x7ff3f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:24:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.772+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3ec0546a0 con 0x7ff3f806c000 2023-11-24T06:24:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.969+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3f80d09b0 con 0x7ff3f806c000 2023-11-24T06:24:59.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.970+0000 7ff3e9ffb700 1 -- 172.21.15.114:0/934128456 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16790 (secure 0 0 0) 0x7ff3ec057cc0 con 0x7ff3f806c000 2023-11-24T06:24:59.972 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:24:59.972 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:59.029214+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:24:59.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 msgr2=0x7ff3dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 0x7ff3dc06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8072a90 tx=0x7ff3f4008030 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 msgr2=0x7ff3f813eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c7f0 tx=0x7ff3ec00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 shutdown_connections 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff3dc06aed0 0x7ff3dc06d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f815e080 0x7ff3f8158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3f806c000 0x7ff3f813eb80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 --2- 172.21.15.114:0/934128456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3f80af4d0 0x7ff3f813f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 >> 172.21.15.114:0/934128456 conn(0x7ff3f8071b90 msgr2=0x7ff3f80b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 shutdown_connections 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:24:59.972+0000 7ff3ff41b700 1 -- 172.21.15.114:0/934128456 wait complete. 2023-11-24T06:24:59.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:24:59.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:00.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 Processor -- start 2023-11-24T06:25:00.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 -- start start 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 0x7fc3a80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3a80afc90 0x7fc3a8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80b3c60 con 0x7fc3a80744d0 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a80b4500 con 0x7fc3a80afc90 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.110+0000 7fc3af136700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8074960 con 0x7fc3a815e160 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3ad6d3700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc3a80744d0 0x7fc3a806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48438/0 (socket says 172.21.15.114:48438) 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3ad6d3700 1 -- 172.21.15.114:0/1628351335 learned_addr learned my addr 172.21.15.114:0/1628351335 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:00.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719437260 0 0) 0x7fc3a80b3c60 con 0x7fc3a80744d0 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3980035b0 con 0x7fc3a80744d0 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2239699201 0 0) 0x7fc3980035b0 con 0x7fc3a80744d0 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 msgr2=0x7fc3a80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 0x7fc3a80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 >> v2:172.21.15.172:3300/0 conn(0x7fc3a80afc90 msgr2=0x7fc3a8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 --2- 172.21.15.114:0/1628351335 >> v2:172.21.15.172:3300/0 conn(0x7fc3a80afc90 0x7fc3a8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80b58c0 con 0x7fc3a80744d0 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.1 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fc3a00019f0 con 0x7fc3a80744d0 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.111+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 >> v1:172.21.15.172:6789/0 conn(0x7fc3a80744d0 legacy=0x7fc3a806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398003a00 0x7fc398003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc398004850 0x7fc398004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 0x7fc39800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b4500 con 0x7fc398004850 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8074960 con 0x7fc398003a00 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980055f0 con 0x7fc3a815e160 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc398004850 0x7fc398004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3aced2700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398003a00 0x7fc398003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 msgr2=0x7fc39800ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a815e160 0x7fc39800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398003a00 msgr2=0x7fc398003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398003a00 0x7fc398003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1628351335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980035b0 con 0x7fc398004850 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc398004850 0x7fc398004cd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc398005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de943f89b36170b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011620 con 0x7fc398004850 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394011780 con 0x7fc398004850 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.112+0000 7fc3a77fe700 1 -- 172.21.15.114:0/1628351335 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011b10 con 0x7fc398004850 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 -- 172.21.15.114:0/1628351335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc398004850 msgr2=0x7fc398004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 --2- 172.21.15.114:0/1628351335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc398004850 0x7fc398004cd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc398005880 comp rx=0 tx=0).stop 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3ad6d3700 1 -- 172.21.15.114:0/1628351335 reap_dead start 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 -- 172.21.15.114:0/1628351335 shutdown_connections 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 -- 172.21.15.114:0/1628351335 >> 172.21.15.114:0/1628351335 conn(0x7fc3a80d0bf0 msgr2=0x7fc3a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 -- 172.21.15.114:0/1628351335 shutdown_connections 2023-11-24T06:25:00.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.113+0000 7fc3af136700 1 -- 172.21.15.114:0/1628351335 wait complete. 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.114+0000 7fc3af136700 1 Processor -- start 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.114+0000 7fc3af136700 1 -- start start 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398004850 0x7fc3a8161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 0x7fc3a81582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8156660 con 0x7fc3a81579a0 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8156900 con 0x7fc398004850 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3af136700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8156be0 con 0x7fc3a8148550 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39148/0 (socket says 172.21.15.114:39148) 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1707910186 learned_addr learned my addr 172.21.15.114:0/1707910186 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3ad6d3700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 0x7fc3a81582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3aced2700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398004850 0x7fc3a8161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 msgr2=0x7fc3a81582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 0x7fc3a81582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398004850 msgr2=0x7fc3a8161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398004850 0x7fc3a8161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 -- 172.21.15.114:0/1707910186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8156ec0 con 0x7fc3a81579a0 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3ad6d3700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 0x7fc3a81582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.115+0000 7fc3a7fff700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc394000c00 tx=0x7fc394004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.116+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394004450 con 0x7fc3a81579a0 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.116+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39401e070 con 0x7fc3a81579a0 2023-11-24T06:25:00.124 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.116+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a81571a0 con 0x7fc3a81579a0 2023-11-24T06:25:00.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.116+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940038c0 con 0x7fc3a81579a0 2023-11-24T06:25:00.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.116+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a8148a00 con 0x7fc3a81579a0 2023-11-24T06:25:00.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.117+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a8148bf0 con 0x7fc3a81579a0 2023-11-24T06:25:00.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.117+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc394003a20 con 0x7fc3a81579a0 2023-11-24T06:25:00.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.117+0000 7fc3a57fa700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.118+0000 7fc3aced2700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.118+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc39408a0b0 con 0x7fc3a81579a0 2023-11-24T06:25:00.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.118+0000 7fc3aced2700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 0x7fc38c06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc39c004790 tx=0x7fc39c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.122+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc394055220 con 0x7fc3a81579a0 2023-11-24T06:25:00.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.310+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3a80b5a80 con 0x7fc3a81579a0 2023-11-24T06:25:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.310+0000 7fc3a57fa700 1 -- 172.21.15.114:0/1707910186 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc394058840 con 0x7fc3a81579a0 2023-11-24T06:25:00.313 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:00.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 msgr2=0x7fc38c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 0x7fc38c06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc39c004790 tx=0x7fc39c008040 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 msgr2=0x7fc3a8156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc394000c00 tx=0x7fc394004860 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 shutdown_connections 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3a8148550 0x7fc3a81582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc398004850 0x7fc3a8161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 --2- 172.21.15.114:0/1707910186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3a81579a0 0x7fc3a8156120 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.312+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 >> 172.21.15.114:0/1707910186 conn(0x7fc3a80d0bf0 msgr2=0x7fc3a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.313+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 shutdown_connections 2023-11-24T06:25:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.313+0000 7fc3af136700 1 -- 172.21.15.114:0/1707910186 wait complete. 2023-11-24T06:25:00.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:00.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.448+0000 7f12b3bc7700 1 Processor -- start 2023-11-24T06:25:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.448+0000 7f12b3bc7700 1 -- start start 2023-11-24T06:25:00.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b3bc7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f12ac15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b3bc7700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac0ae5e0 con 0x7f12ac15b370 2023-11-24T06:25:00.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b3bc7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac15be30 con 0x7f12ac1587f0 2023-11-24T06:25:00.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b3bc7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac15c110 con 0x7f12ac0b4370 2023-11-24T06:25:00.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b1162700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f12ac15b370 0x7f12ac0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54862/0 (socket says 172.21.15.114:54862) 2023-11-24T06:25:00.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b1162700 1 -- 172.21.15.114:0/2529171727 learned_addr learned my addr 172.21.15.114:0/2529171727 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:00.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b2164700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f12ac15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596995 0 0) 0x7f12ac0ae5e0 con 0x7f12ac15b370 2023-11-24T06:25:00.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.449+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12980035b0 con 0x7f12ac15b370 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753037182 0 0) 0x7f12980035b0 con 0x7f12ac15b370 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 msgr2=0x7f12ac15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f12ac15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 >> v1:172.21.15.172:6789/0 conn(0x7f12ac1587f0 legacy=0x7f12ac06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0ae860 con 0x7f12ac15b370 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f12a8001670 con 0x7f12ac15b370 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 >> v1:172.21.15.114:6789/0 conn(0x7f12ac15b370 legacy=0x7f12ac0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1298003a30 0x7f1298003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f1298004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f1298005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac15c110 con 0x7f1298004850 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129800baf0 con 0x7f1298003a30 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129800dec0 con 0x7f12ac0b4370 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f1298004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1963700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1298003a30 0x7f1298003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b2164700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f1298005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 -- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 msgr2=0x7f1298005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac0b4370 0x7f1298005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 -- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1298003a30 msgr2=0x7f1298003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1298003a30 0x7f1298003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 -- 172.21.15.114:0/2529171727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980035b0 con 0x7f1298004850 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b1162700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f1298004cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c210 tx=0x7f1298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482688c39320e39e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.450+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8007c10 con 0x7f1298004850 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a800cdb0 con 0x7f1298004850 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b0961700 1 -- 172.21.15.114:0/2529171727 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8005ba0 con 0x7f1298004850 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2529171727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 msgr2=0x7f1298004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2529171727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f1298004cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c210 tx=0x7f1298005880 comp rx=0 tx=0).stop 2023-11-24T06:25:00.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b2164700 1 -- 172.21.15.114:0/2529171727 reap_dead start 2023-11-24T06:25:00.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2529171727 shutdown_connections 2023-11-24T06:25:00.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2529171727 >> 172.21.15.114:0/2529171727 conn(0x7f12ac0d0bf0 msgr2=0x7f12ac0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2529171727 shutdown_connections 2023-11-24T06:25:00.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.451+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2529171727 wait complete. 2023-11-24T06:25:00.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.452+0000 7f12b3bc7700 1 Processor -- start 2023-11-24T06:25:00.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.452+0000 7f12b3bc7700 1 -- start start 2023-11-24T06:25:00.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac154d10 0x7f12ac0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 0x7f12ac071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac09d3c0 con 0x7f1298004850 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0b11a0 con 0x7f12ac154d10 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b3bc7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0b1960 con 0x7f12ac1544f0 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39166/0 (socket says 172.21.15.114:39166) 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 -- 172.21.15.114:0/2957442255 learned_addr learned my addr 172.21.15.114:0/2957442255 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1162700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac154d10 0x7f12ac0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 -- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 msgr2=0x7f12ac071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b2164700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 0x7f12ac071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 0x7f12ac071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 -- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac154d10 msgr2=0x7f12ac0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac154d10 0x7f12ac0b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b1963700 1 -- 172.21.15.114:0/2957442255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0b0180 con 0x7f1298004850 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.453+0000 7f12b2164700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 0x7f12ac071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12b1963700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f129c00f7f0 tx=0x7f129c00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0052e0 con 0x7f1298004850 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c005440 con 0x7f1298004850 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac0b0460 con 0x7f1298004850 2023-11-24T06:25:00.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0038a0 con 0x7f1298004850 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.454+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac0b1c40 con 0x7f1298004850 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.455+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac0b0ab0 con 0x7f1298004850 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.455+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f129c003a00 con 0x7f1298004850 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.456+0000 7f12a27fc700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 0x7f129006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.456+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f129c089380 con 0x7f1298004850 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.456+0000 7f12b1162700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 0x7f129006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.456+0000 7f12b1162700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 0x7f129006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12a8015040 tx=0x7f12a8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.460+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f129c054440 con 0x7f1298004850 2023-11-24T06:25:00.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.649+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12ac06bf20 con 0x7f1298004850 2023-11-24T06:25:00.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.650+0000 7f12a27fc700 1 -- 172.21.15.114:0/2957442255 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f129c003cb0 con 0x7f1298004850 2023-11-24T06:25:00.652 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:00.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.651+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 msgr2=0x7f129006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.651+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 0x7f129006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12a8015040 tx=0x7f12a8007000 comp rx=0 tx=0).stop 2023-11-24T06:25:00.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 msgr2=0x7f12ac09e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f129c00f7f0 tx=0x7f129c00fbb0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 shutdown_connections 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f129006aed0 0x7f129006d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12ac1544f0 0x7f12ac071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1298004850 0x7f12ac09e790 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 --2- 172.21.15.114:0/2957442255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac154d10 0x7f12ac0b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 >> 172.21.15.114:0/2957442255 conn(0x7f12ac0d0bf0 msgr2=0x7f12ac15a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 shutdown_connections 2023-11-24T06:25:00.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.652+0000 7f12b3bc7700 1 -- 172.21.15.114:0/2957442255 wait complete. 2023-11-24T06:25:00.666 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:00.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 Processor -- start 2023-11-24T06:25:00.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 -- start start 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9cc15bec0 0x7fb9cc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 0x7fb9cc0af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0cc4a0 con 0x7fb9cc0702e0 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc070770 con 0x7fb9cc15bec0 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.787+0000 7fb9d2052700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0d03e0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 0x7fb9cc0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9caffd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9cc0702e0 0x7fb9cc0d1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48480/0 (socket says 172.21.15.114:48480) 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9caffd700 1 -- 172.21.15.114:0/1279863403 learned_addr learned my addr 172.21.15.114:0/1279863403 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:00.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 -- 172.21.15.114:0/1279863403 >> v2:172.21.15.172:3300/0 conn(0x7fb9cc15bec0 msgr2=0x7fb9cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 --2- 172.21.15.114:0/1279863403 >> v2:172.21.15.172:3300/0 conn(0x7fb9cc15bec0 0x7fb9cc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 -- 172.21.15.114:0/1279863403 >> v1:172.21.15.172:6789/0 conn(0x7fb9cc0702e0 legacy=0x7fb9cc0d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 -- 172.21.15.114:0/1279863403 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc15c2a0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9cbfff700 1 --2- 172.21.15.114:0/1279863403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 0x7fb9cc0af520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009d90 tx=0x7fb9c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0b8899823c674a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9ca7fc700 1 -- 172.21.15.114:0/1279863403 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400e070 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9ca7fc700 1 -- 172.21.15.114:0/1279863403 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400bbd0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.788+0000 7fb9ca7fc700 1 -- 172.21.15.114:0/1279863403 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4012430 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 -- 172.21.15.114:0/1279863403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 msgr2=0x7fb9cc0af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 --2- 172.21.15.114:0/1279863403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 0x7fb9cc0af520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009d90 tx=0x7fb9c400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 -- 172.21.15.114:0/1279863403 shutdown_connections 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 --2- 172.21.15.114:0/1279863403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0d21d0 0x7fb9cc0af520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 --2- 172.21.15.114:0/1279863403 >> v2:172.21.15.172:3300/0 conn(0x7fb9cc15bec0 0x7fb9cc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 -- 172.21.15.114:0/1279863403 >> 172.21.15.114:0/1279863403 conn(0x7fb9cc0730a0 msgr2=0x7fb9cc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 -- 172.21.15.114:0/1279863403 shutdown_connections 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.789+0000 7fb9d2052700 1 -- 172.21.15.114:0/1279863403 wait complete. 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.790+0000 7fb9d2052700 1 Processor -- start 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.790+0000 7fb9d2052700 1 -- start start 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0702e0 0x7fb9cc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9cc15bec0 0x7fb9cc15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15d510 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc06c8e0 con 0x7fb9cc15bec0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9d2052700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15c530 con 0x7fb9cc0702e0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39182/0 (socket says 172.21.15.114:39182) 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9caffd700 1 -- 172.21.15.114:0/882555966 learned_addr learned my addr 172.21.15.114:0/882555966 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9cb7fe700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0702e0 0x7fb9cc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.791+0000 7fb9cbfff700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9cc15bec0 0x7fb9cc15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 -- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0702e0 msgr2=0x7fb9cc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0702e0 0x7fb9cc15cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 -- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9cc15bec0 msgr2=0x7fb9cc15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9cc15bec0 0x7fb9cc15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 -- 172.21.15.114:0/882555966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009a70 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9caffd700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c480 tx=0x7fb9bc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc005d70 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0070d0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc15e870 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc007440 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.792+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc15c880 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.793+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc068fb0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.793+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9bc0075a0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.794+0000 7fb9b3fff700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 0x7fb9b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.794+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb9bc088a50 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.794+0000 7fb9cb7fe700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 0x7fb9b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:00.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.794+0000 7fb9cb7fe700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 0x7fb9b806d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0006fd0 tx=0x7fb9c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:00.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.798+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9bc053bc0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.985+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9cc15f4c0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.986+0000 7fb9b3fff700 1 -- 172.21.15.114:0/882555966 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9bc0170a0 con 0x7fb9cc0d21d0 2023-11-24T06:25:00.989 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 msgr2=0x7fb9b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 0x7fb9b806d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0006fd0 tx=0x7fb9c0008040 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 msgr2=0x7fb9cc0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c480 tx=0x7fb9bc00c840 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 shutdown_connections 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9b806aed0 0x7fb9b806d390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9cc15bec0 0x7fb9cc15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9cc0702e0 0x7fb9cc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 --2- 172.21.15.114:0/882555966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9cc0d21d0 0x7fb9cc0ce3b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:00.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 >> 172.21.15.114:0/882555966 conn(0x7fb9cc0730a0 msgr2=0x7fb9cc0b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:00.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 shutdown_connections 2023-11-24T06:25:00.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:00.988+0000 7fb9d2052700 1 -- 172.21.15.114:0/882555966 wait complete. 2023-11-24T06:25:01.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.122+0000 7effb25c8700 1 Processor -- start 2023-11-24T06:25:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.122+0000 7effb25c8700 1 -- start start 2023-11-24T06:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effb25c8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effac15e080 0x7effac0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effb25c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac06c710 0x7effac0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effb25c8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effac0ae430 con 0x7effac0af4d0 2023-11-24T06:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effb25c8700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effac0af0b0 con 0x7effac15e080 2023-11-24T06:25:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effb25c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac0b05a0 con 0x7effac06c710 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effabfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effac15e080 0x7effac0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effabfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7effac15e080 0x7effac0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39188/0 (socket says 172.21.15.114:39188) 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.123+0000 7effabfff700 1 -- 172.21.15.114:0/1064208835 learned_addr learned my addr 172.21.15.114:0/1064208835 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effb0b65700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac06c710 0x7effac0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effabfff700 1 -- 172.21.15.114:0/1064208835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac06c710 msgr2=0x7effac0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effabfff700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac06c710 0x7effac0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effabfff700 1 -- 172.21.15.114:0/1064208835 >> v1:172.21.15.172:6789/0 conn(0x7effac0af4d0 legacy=0x7effac06c000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effabfff700 1 -- 172.21.15.114:0/1064208835 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac0ae710 con 0x7effac15e080 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effabfff700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac15e080 0x7effac0b5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effa0009920 tx=0x7effa0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dec460f8e26d0f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effaaffd700 1 -- 172.21.15.114:0/1064208835 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0004230 con 0x7effac15e080 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effaaffd700 1 -- 172.21.15.114:0/1064208835 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0004390 con 0x7effac15e080 2023-11-24T06:25:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.124+0000 7effaaffd700 1 -- 172.21.15.114:0/1064208835 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa00036a0 con 0x7effac15e080 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 -- 172.21.15.114:0/1064208835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac15e080 msgr2=0x7effac0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac15e080 0x7effac0b5050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effa0009920 tx=0x7effa0009c30 comp rx=0 tx=0).stop 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 -- 172.21.15.114:0/1064208835 shutdown_connections 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac06c710 0x7effac0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 --2- 172.21.15.114:0/1064208835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac15e080 0x7effac0b5050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 -- 172.21.15.114:0/1064208835 >> 172.21.15.114:0/1064208835 conn(0x7effac071b90 msgr2=0x7effac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 -- 172.21.15.114:0/1064208835 shutdown_connections 2023-11-24T06:25:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.125+0000 7effb25c8700 1 -- 172.21.15.114:0/1064208835 wait complete. 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.126+0000 7effb25c8700 1 Processor -- start 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.126+0000 7effb25c8700 1 -- start start 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effac06c710 0x7effac159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac15e080 0x7effac14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac15a0a0 con 0x7effac0af4d0 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac1f51e0 con 0x7effac06c710 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effb25c8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac1583d0 con 0x7effac15e080 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39196/0 (socket says 172.21.15.114:39196) 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 -- 172.21.15.114:0/2119649140 learned_addr learned my addr 172.21.15.114:0/2119649140 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 -- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac15e080 msgr2=0x7effac14a3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac15e080 0x7effac14a3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 -- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effac06c710 msgr2=0x7effac159b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.127+0000 7effab7fe700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effac06c710 0x7effac159b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effab7fe700 1 -- 172.21.15.114:0/2119649140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0009550 con 0x7effac0af4d0 2023-11-24T06:25:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effab7fe700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b550 tx=0x7eff9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011830 con 0x7effac0af4d0 2023-11-24T06:25:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c0119b0 con 0x7effac0af4d0 2023-11-24T06:25:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac1f46d0 con 0x7effac0af4d0 2023-11-24T06:25:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011d60 con 0x7effac0af4d0 2023-11-24T06:25:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.128+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac1f4c20 con 0x7effac0af4d0 2023-11-24T06:25:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.129+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac0753f0 con 0x7effac0af4d0 2023-11-24T06:25:01.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.129+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff9c01a400 con 0x7effac0af4d0 2023-11-24T06:25:01.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.130+0000 7effa8ff9700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 0x7eff9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.130+0000 7effabfff700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 0x7eff9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.130+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7eff9c0895b0 con 0x7effac0af4d0 2023-11-24T06:25:01.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.130+0000 7effabfff700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 0x7eff9806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effa0009c70 tx=0x7effa0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.134+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff9c054670 con 0x7effac0af4d0 2023-11-24T06:25:01.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.339+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7effac067f60 con 0x7effac0af4d0 2023-11-24T06:25:01.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.340+0000 7effa8ff9700 1 -- 172.21.15.114:0/2119649140 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff9c057c90 con 0x7effac0af4d0 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 msgr2=0x7eff9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 0x7eff9806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effa0009c70 tx=0x7effa0008030 comp rx=0 tx=0).stop 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 msgr2=0x7effac1f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b550 tx=0x7eff9c00b910 comp rx=0 tx=0).stop 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 shutdown_connections 2023-11-24T06:25:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9806aed0 0x7eff9806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effac15e080 0x7effac14a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effac06c710 0x7effac159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 --2- 172.21.15.114:0/2119649140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effac0af4d0 0x7effac1f4190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.341+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 >> 172.21.15.114:0/2119649140 conn(0x7effac071b90 msgr2=0x7effac0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.342+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 shutdown_connections 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.342+0000 7effb25c8700 1 -- 172.21.15.114:0/2119649140 wait complete. 2023-11-24T06:25:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:25:01.356 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-24T06:25:01.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:01.356 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-24T06:25:01.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:25:01.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:01.394 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:25:01.451 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:01.451 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:01.473 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:01.473 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:01.533 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:01.534 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:01.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:01.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.713+0000 7f7d3276b700 1 Processor -- start 2023-11-24T06:25:01.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.713+0000 7f7d3276b700 1 -- start start 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d3276b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 0x7f7d2c0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d3276b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d2c0cf360 0x7f7d2c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d3276b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c15a790 con 0x7f7d2c0b3da0 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d3276b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c15eec0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d3276b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0cf8b0 con 0x7f7d2c15a3b0 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d2c0cf360 0x7f7d2c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d2c0cf360 0x7f7d2c0b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39204/0 (socket says 172.21.15.114:39204) 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/1320111906 learned_addr learned my addr 172.21.15.114:0/1320111906 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2bfff700 1 --2- 172.21.15.114:0/1320111906 >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 0x7f7d2c0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/1320111906 >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 msgr2=0x7f7d2c0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 --2- 172.21.15.114:0/1320111906 >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 0x7f7d2c0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/1320111906 >> v1:172.21.15.172:6789/0 conn(0x7f7d2c0b3da0 legacy=0x7f7d2c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/1320111906 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0b4c40 con 0x7f7d2c0cf360 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2bfff700 1 --2- 172.21.15.114:0/1320111906 >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 0x7f7d2c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2b7fe700 1 --2- 172.21.15.114:0/1320111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c0b3860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c04fca0 tx=0x7f7d1c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b9e1666813a486 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2affd700 1 -- 172.21.15.114:0/1320111906 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00dc00 con 0x7f7d2c0cf360 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.714+0000 7f7d2affd700 1 -- 172.21.15.114:0/1320111906 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00dd60 con 0x7f7d2c0cf360 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d2affd700 1 -- 172.21.15.114:0/1320111906 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004270 con 0x7f7d2c0cf360 2023-11-24T06:25:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 -- 172.21.15.114:0/1320111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 msgr2=0x7f7d2c0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 --2- 172.21.15.114:0/1320111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c0b3860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c04fca0 tx=0x7f7d1c00bea0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 -- 172.21.15.114:0/1320111906 shutdown_connections 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 --2- 172.21.15.114:0/1320111906 >> v2:172.21.15.172:3300/0 conn(0x7f7d2c15a3b0 0x7f7d2c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 --2- 172.21.15.114:0/1320111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c0b3860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 -- 172.21.15.114:0/1320111906 >> 172.21.15.114:0/1320111906 conn(0x7f7d2c0b1740 msgr2=0x7f7d2c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 -- 172.21.15.114:0/1320111906 shutdown_connections 2023-11-24T06:25:01.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.715+0000 7f7d3276b700 1 -- 172.21.15.114:0/1320111906 wait complete. 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.716+0000 7f7d3276b700 1 Processor -- start 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.716+0000 7f7d3276b700 1 -- start start 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d2c0b3da0 0x7f7d2c149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d2c15a3b0 0x7f7d2c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c154420 con 0x7f7d2c0cf360 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c154700 con 0x7f7d2c15a3b0 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d3276b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1549e0 con 0x7f7d2c0b3da0 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39210/0 (socket says 172.21.15.114:39210) 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/3540409127 learned_addr learned my addr 172.21.15.114:0/3540409127 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.717+0000 7f7d2bfff700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d2c0b3da0 0x7f7d2c149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d2c0b3da0 msgr2=0x7f7d2c149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d2c0b3da0 0x7f7d2c149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d2c15a3b0 msgr2=0x7f7d2c156290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d2c15a3b0 0x7f7d2c156290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 -- 172.21.15.114:0/3540409127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c00b7f0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d2b7fe700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c019040 tx=0x7f7d1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009040 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0045a0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c154cc0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004270 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.718+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c1567d0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.719+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c0751e0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.720+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d1c015070 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.720+0000 7f7d28ff9700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 0x7f7d1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.720+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7d1c08b470 con 0x7f7d2c0cf360 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.720+0000 7f7d2bfff700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 0x7f7d1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:01.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.720+0000 7f7d2bfff700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 0x7f7d1806d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:01.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.724+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d1c056650 con 0x7f7d2c0cf360 2023-11-24T06:25:01.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.916+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d2c1551e0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.917+0000 7f7d28ff9700 1 -- 172.21.15.114:0/3540409127 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f7d2c1551e0 con 0x7f7d2c0cf360 2023-11-24T06:25:01.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:01.920 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:59.029214+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 msgr2=0x7f7d1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 0x7f7d1806d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20008040 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 msgr2=0x7f7d2c14d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c019040 tx=0x7f7d1c004750 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 shutdown_connections 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7d1806aed0 0x7f7d1806d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d2c15a3b0 0x7f7d2c156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d2c0b3da0 0x7f7d2c149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 --2- 172.21.15.114:0/3540409127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d2c0cf360 0x7f7d2c14d750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 >> 172.21.15.114:0/3540409127 conn(0x7f7d2c0b1740 msgr2=0x7f7d2c157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 shutdown_connections 2023-11-24T06:25:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:01.919+0000 7f7d3276b700 1 -- 172.21.15.114:0/3540409127 wait complete. 2023-11-24T06:25:01.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:01.934 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:02.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.055+0000 7f7a3c191700 1 Processor -- start 2023-11-24T06:25:02.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.055+0000 7f7a3c191700 1 -- start start 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a3c191700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7a340746a0 0x7f7a340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a3c191700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340b3c60 con 0x7f7a3415e160 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a3c191700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340b58c0 con 0x7f7a340afc90 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a3c191700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b4500 con 0x7f7a340746a0 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a39f2d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7a3415e160 0x7f7a34073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54924/0 (socket says 172.21.15.114:54924) 2023-11-24T06:25:02.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a39f2d700 1 -- 172.21.15.114:0/486657847 learned_addr learned my addr 172.21.15.114:0/486657847 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103522072 0 0) 0x7f7a340b3c60 con 0x7f7a3415e160 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a200035b0 con 0x7f7a3415e160 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a3972c700 1 --2- 172.21.15.114:0/486657847 >> v2:172.21.15.172:3300/0 conn(0x7f7a340746a0 0x7f7a340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.056+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363337855 0 0) 0x7f7a200035b0 con 0x7f7a3415e160 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 >> v2:172.21.15.172:3300/0 conn(0x7f7a340746a0 msgr2=0x7f7a340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 --2- 172.21.15.114:0/486657847 >> v2:172.21.15.172:3300/0 conn(0x7f7a340746a0 0x7f7a340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 >> v1:172.21.15.172:6789/0 conn(0x7f7a340afc90 legacy=0x7f7a3406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3406bdc0 con 0x7f7a3415e160 2023-11-24T06:25:02.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f7a24001670 con 0x7f7a3415e160 2023-11-24T06:25:02.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 >> v1:172.21.15.114:6789/0 conn(0x7f7a3415e160 legacy=0x7f7a34073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a20003a00 0x7f7a20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b4500 con 0x7f7a20005210 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2000c140 con 0x7f7a20004850 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2000c300 con 0x7f7a20003a00 2023-11-24T06:25:02.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 -- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a20003a00 msgr2=0x7f7a20003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a20003a00 0x7f7a20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 -- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a20004850 msgr2=0x7f7a20004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 -- 172.21.15.114:0/486657847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200035b0 con 0x7f7a20005210 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 -- 172.21.15.114:0/486657847 reap_dead start 2023-11-24T06:25:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a3a72e700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a2000ba10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c001db0 tx=0x7f7a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7030c1cae144a101 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c001850 con 0x7f7a20005210 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c003c20 con 0x7f7a20005210 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.057+0000 7f7a38f2b700 1 -- 172.21.15.114:0/486657847 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0050d0 con 0x7f7a20005210 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 -- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 msgr2=0x7f7a2000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a2000ba10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c001db0 tx=0x7f7a20005880 comp rx=0 tx=0).stop 2023-11-24T06:25:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 -- 172.21.15.114:0/486657847 shutdown_connections 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 --2- 172.21.15.114:0/486657847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 -- 172.21.15.114:0/486657847 >> 172.21.15.114:0/486657847 conn(0x7f7a340d0bf0 msgr2=0x7f7a340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 -- 172.21.15.114:0/486657847 shutdown_connections 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.058+0000 7f7a3c191700 1 -- 172.21.15.114:0/486657847 wait complete. 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.059+0000 7f7a3c191700 1 Processor -- start 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.059+0000 7f7a3c191700 1 -- start start 2023-11-24T06:25:02.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 0x7f7a340c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a3415f940 0x7f7a3415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34149510 con 0x7f7a20005210 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341f1680 con 0x7f7a340c8690 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3c191700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340c9050 con 0x7f7a3415f940 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39228/0 (socket says 172.21.15.114:39228) 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 -- 172.21.15.114:0/1128654843 learned_addr learned my addr 172.21.15.114:0/1128654843 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3972c700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 0x7f7a340c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3a72e700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a3415f940 0x7f7a3415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 -- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a3415f940 msgr2=0x7f7a3415fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a3415f940 0x7f7a3415fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 -- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 msgr2=0x7f7a340c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 0x7f7a340c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 -- 172.21.15.114:0/1128654843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34160670 con 0x7f7a20005210 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a3972c700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 0x7f7a340c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a39f2d700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400cad0 tx=0x7f7a2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.060+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24004a80 con 0x7f7a20005210 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.061+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a24004be0 con 0x7f7a20005210 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.061+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34160950 con 0x7f7a20005210 2023-11-24T06:25:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.061+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a240070d0 con 0x7f7a20005210 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.061+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34160f10 con 0x7f7a20005210 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.062+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34068fb0 con 0x7f7a20005210 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.062+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a24007230 con 0x7f7a20005210 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.062+0000 7f7a2a7fc700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 0x7f7a1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.062+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7a24088d20 con 0x7f7a20005210 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.062+0000 7f7a3972c700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 0x7f7a1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.063+0000 7f7a3972c700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 0x7f7a1806d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006b20 tx=0x7f7a30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.067+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a24053d40 con 0x7f7a20005210 2023-11-24T06:25:02.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.258+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a34149650 con 0x7f7a20005210 2023-11-24T06:25:02.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.259+0000 7f7a2a7fc700 1 -- 172.21.15.114:0/1128654843 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a34149650 con 0x7f7a20005210 2023-11-24T06:25:02.263 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:02.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 msgr2=0x7f7a1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 0x7f7a1806d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006b20 tx=0x7f7a30008040 comp rx=0 tx=0).stop 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 msgr2=0x7f7a341f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400cad0 tx=0x7f7a2400cde0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 shutdown_connections 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7a1806aed0 0x7f7a1806d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a3415f940 0x7f7a3415fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20005210 0x7f7a341f1140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 --2- 172.21.15.114:0/1128654843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a340c8690 0x7f7a340c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 >> 172.21.15.114:0/1128654843 conn(0x7f7a340d0bf0 msgr2=0x7f7a340b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 shutdown_connections 2023-11-24T06:25:02.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.261+0000 7f7a3c191700 1 -- 172.21.15.114:0/1128654843 wait complete. 2023-11-24T06:25:02.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:02.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.396+0000 7f1c5fc5f700 1 Processor -- start 2023-11-24T06:25:02.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.396+0000 7f1c5fc5f700 1 -- start start 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5fc5f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5815e320 0x7f1c580d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5fc5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c58167d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5fc5f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c580b6fb0 con 0x7f1c580b93c0 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5fc5f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c580d4da0 con 0x7f1c5815e320 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5fc5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c580d2940 con 0x7f1c5806b700 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5815e320 0x7f1c580d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d1fa700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1c580b93c0 0x7f1c5806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54944/0 (socket says 172.21.15.114:54944) 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c5815e320 0x7f1c580d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39236/0 (socket says 172.21.15.114:39236) 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d1fa700 1 -- 172.21.15.114:0/3278837105 learned_addr learned my addr 172.21.15.114:0/3278837105 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:02.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5e1fc700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c58167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 -- 172.21.15.114:0/3278837105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 msgr2=0x7f1c58167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c58167d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 -- 172.21.15.114:0/3278837105 >> v1:172.21.15.114:6789/0 conn(0x7f1c580b93c0 legacy=0x7f1c5806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5d9fb700 1 -- 172.21.15.114:0/3278837105 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580d2c10 con 0x7f1c5815e320 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5c9f9700 1 -- 172.21.15.114:0/3278837105 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274306388 0 0) 0x7f1c580b6fb0 con 0x7f1c580b93c0 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.397+0000 7f1c5e1fc700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c58167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5d9fb700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c580d2400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009840 tx=0x7f1c48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4e74c3486f21ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5c9f9700 1 -- 172.21.15.114:0/3278837105 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48011830 con 0x7f1c5815e320 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5c9f9700 1 -- 172.21.15.114:0/3278837105 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c48011990 con 0x7f1c5815e320 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5c9f9700 1 -- 172.21.15.114:0/3278837105 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48011d20 con 0x7f1c5815e320 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/3278837105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 msgr2=0x7f1c580d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c580d2400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009840 tx=0x7f1c48009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/3278837105 shutdown_connections 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c58167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/3278837105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c580d2400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/3278837105 >> 172.21.15.114:0/3278837105 conn(0x7f1c580d3e50 msgr2=0x7f1c580d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/3278837105 shutdown_connections 2023-11-24T06:25:02.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.398+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/3278837105 wait complete. 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 Processor -- start 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 -- start start 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c581e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 0x7f1c580c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581f3c10 con 0x7f1c5815e320 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581e6cc0 con 0x7f1c580b93c0 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5fc5f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58145120 con 0x7f1c5806b700 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5e1fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5e1fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39246/0 (socket says 172.21.15.114:39246) 2023-11-24T06:25:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.400+0000 7f1c5e1fc700 1 -- 172.21.15.114:0/1749072090 learned_addr learned my addr 172.21.15.114:0/1749072090 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5d9fb700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c581e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5d1fa700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 0x7f1c580c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 -- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 msgr2=0x7f1c581e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c581e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 -- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 msgr2=0x7f1c580c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 0x7f1c580c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 -- 172.21.15.114:0/1749072090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480094a0 con 0x7f1c5815e320 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5d1fa700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 0x7f1c580c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5e1fc700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000cef0 tx=0x7f1c5000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000adf0 con 0x7f1c5815e320 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50015440 con 0x7f1c5815e320 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c58145400 con 0x7f1c5815e320 2023-11-24T06:25:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500157f0 con 0x7f1c5815e320 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.401+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c581459c0 con 0x7f1c5815e320 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.402+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c580cfbd0 con 0x7f1c5815e320 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.402+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c5000d430 con 0x7f1c5815e320 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.403+0000 7f1c4e7fc700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 0x7f1c3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.403+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1c50089bd0 con 0x7f1c5815e320 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.403+0000 7f1c5d9fb700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 0x7f1c3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.403+0000 7f1c5d9fb700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 0x7f1c3c06d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1c480090b0 tx=0x7f1c4801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.407+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c50054b70 con 0x7f1c5815e320 2023-11-24T06:25:02.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.595+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c58068fb0 con 0x7f1c5815e320 2023-11-24T06:25:02.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.596+0000 7f1c4e7fc700 1 -- 172.21.15.114:0/1749072090 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c5001a020 con 0x7f1c5815e320 2023-11-24T06:25:02.598 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 msgr2=0x7f1c3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 0x7f1c3c06d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1c480090b0 tx=0x7f1c4801c040 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 msgr2=0x7f1c581591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000cef0 tx=0x7f1c5000cf20 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 shutdown_connections 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c3c06af00 0x7f1c3c06d3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c5815e320 0x7f1c581591b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5806b700 0x7f1c581e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 --2- 172.21.15.114:0/1749072090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c580b93c0 0x7f1c580c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 >> 172.21.15.114:0/1749072090 conn(0x7f1c580d3e50 msgr2=0x7f1c580b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 shutdown_connections 2023-11-24T06:25:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.598+0000 7f1c5fc5f700 1 -- 172.21.15.114:0/1749072090 wait complete. 2023-11-24T06:25:02.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.733+0000 7f89a42c4700 1 Processor -- start 2023-11-24T06:25:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.733+0000 7f89a42c4700 1 -- start start 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a42c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a42c4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899c15b370 0x7f899c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a42c4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c06b530 con 0x7f899c0b3c60 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a42c4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0ae5e0 con 0x7f899c15b370 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a42c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c15be30 con 0x7f899c1587f0 2023-11-24T06:25:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899c15b370 0x7f899c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899c15b370 0x7f899c15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39250/0 (socket says 172.21.15.114:39250) 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 -- 172.21.15.114:0/743309073 learned_addr learned my addr 172.21.15.114:0/743309073 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 -- 172.21.15.114:0/743309073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 msgr2=0x7f899c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 --2- 172.21.15.114:0/743309073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 -- 172.21.15.114:0/743309073 >> v1:172.21.15.172:6789/0 conn(0x7f899c0b3c60 legacy=0x7f899c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 -- 172.21.15.114:0/743309073 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c15c110 con 0x7f899c15b370 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a185f700 1 --2- 172.21.15.114:0/743309073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c15b370 0x7f899c15b7f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f898c0098f0 tx=0x7f898c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9497db58a9e6ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a105e700 1 -- 172.21.15.114:0/743309073 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c004230 con 0x7f899c15b370 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a105e700 1 -- 172.21.15.114:0/743309073 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c004390 con 0x7f899c15b370 2023-11-24T06:25:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.734+0000 7f89a105e700 1 -- 172.21.15.114:0/743309073 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c0036a0 con 0x7f899c15b370 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 -- 172.21.15.114:0/743309073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c15b370 msgr2=0x7f899c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 --2- 172.21.15.114:0/743309073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c15b370 0x7f899c15b7f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f898c0098f0 tx=0x7f898c009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 -- 172.21.15.114:0/743309073 shutdown_connections 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 --2- 172.21.15.114:0/743309073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 --2- 172.21.15.114:0/743309073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c15b370 0x7f899c15b7f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 -- 172.21.15.114:0/743309073 >> 172.21.15.114:0/743309073 conn(0x7f899c0d0bf0 msgr2=0x7f899c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 -- 172.21.15.114:0/743309073 shutdown_connections 2023-11-24T06:25:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.735+0000 7f89a42c4700 1 -- 172.21.15.114:0/743309073 wait complete. 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.736+0000 7f89a42c4700 1 Processor -- start 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.736+0000 7f89a42c4700 1 -- start start 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c15b370 0x7f899c1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c158f20 con 0x7f899c0b3c60 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c140840 con 0x7f899c15b370 2023-11-24T06:25:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a42c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c140a90 con 0x7f899c1587f0 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39254/0 (socket says 172.21.15.114:39254) 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 -- 172.21.15.114:0/179198559 learned_addr learned my addr 172.21.15.114:0/179198559 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a185f700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2861700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c15b370 0x7f899c1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 -- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 msgr2=0x7f899c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 -- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c15b370 msgr2=0x7f899c1473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c15b370 0x7f899c1473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a2060700 1 -- 172.21.15.114:0/179198559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c009550 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.737+0000 7f89a185f700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f89a2060700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f899800b530 tx=0x7f899800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998011830 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89980119b0 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c140d70 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998011d60 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.738+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c141330 con 0x7f899c0b3c60 2023-11-24T06:25:02.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.739+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c068fb0 con 0x7f899c0b3c60 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.739+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f899801a400 con 0x7f899c0b3c60 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.739+0000 7f8992ffd700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 0x7f898006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.739+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8998089720 con 0x7f899c0b3c60 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.739+0000 7f89a185f700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 0x7f898006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.740+0000 7f89a185f700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 0x7f898006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f898c009130 tx=0x7f898c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:02.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.744+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8998054750 con 0x7f899c0b3c60 2023-11-24T06:25:02.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.936+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f899c074e60 con 0x7f899c0b3c60 2023-11-24T06:25:02.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.937+0000 7f8992ffd700 1 -- 172.21.15.114:0/179198559 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89980150a0 con 0x7f899c0b3c60 2023-11-24T06:25:02.940 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 msgr2=0x7f898006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 0x7f898006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f898c009130 tx=0x7f898c008030 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 msgr2=0x7f899c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f899800b530 tx=0x7f899800b8f0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 shutdown_connections 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f898006aed0 0x7f898006d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c15b370 0x7f899c1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899c0b3c60 0x7f899c15a530 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 --2- 172.21.15.114:0/179198559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f899c1587f0 0x7f899c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:02.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 >> 172.21.15.114:0/179198559 conn(0x7f899c0d0bf0 msgr2=0x7f899c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:02.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 shutdown_connections 2023-11-24T06:25:02.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:02.939+0000 7f89a42c4700 1 -- 172.21.15.114:0/179198559 wait complete. 2023-11-24T06:25:02.954 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:25:03.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.073+0000 7fa6353b1700 1 Processor -- start 2023-11-24T06:25:03.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.073+0000 7fa6353b1700 1 -- start start 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa6353b1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa6300622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa6353b1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6300b04b0 0x7fa630165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa6353b1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63015c540 con 0x7fa6300b2870 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa6353b1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6300b2cf0 con 0x7fa6300b04b0 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa6353b1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300b2ee0 con 0x7fa63015c160 2023-11-24T06:25:03.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6300b04b0 0x7fa630165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62e7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa6300b2870 0x7fa6300afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54970/0 (socket says 172.21.15.114:54970) 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6300b04b0 0x7fa630165c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39266/0 (socket says 172.21.15.114:39266) 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62e7fc700 1 -- 172.21.15.114:0/2369926371 learned_addr learned my addr 172.21.15.114:0/2369926371 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62effd700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa6300622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62f7fe700 1 -- 172.21.15.114:0/2369926371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 msgr2=0x7fa6300622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.074+0000 7fa62f7fe700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa6300622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62f7fe700 1 -- 172.21.15.114:0/2369926371 >> v1:172.21.15.114:6789/0 conn(0x7fa6300b2870 legacy=0x7fa6300afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62f7fe700 1 -- 172.21.15.114:0/2369926371 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300add40 con 0x7fa6300b04b0 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62dffb700 1 -- 172.21.15.114:0/2369926371 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804010631 0 0) 0x7fa63015c540 con 0x7fa6300b2870 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62f7fe700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630165c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa628009d90 tx=0x7fa62800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109ca13973c759d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62dffb700 1 -- 172.21.15.114:0/2369926371 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800e070 con 0x7fa6300b04b0 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62dffb700 1 -- 172.21.15.114:0/2369926371 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800bb20 con 0x7fa6300b04b0 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa62dffb700 1 -- 172.21.15.114:0/2369926371 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628012430 con 0x7fa6300b04b0 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 -- 172.21.15.114:0/2369926371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 msgr2=0x7fa630165c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630165c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa628009d90 tx=0x7fa62800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 -- 172.21.15.114:0/2369926371 shutdown_connections 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa6300622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 --2- 172.21.15.114:0/2369926371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630165c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 -- 172.21.15.114:0/2369926371 >> 172.21.15.114:0/2369926371 conn(0x7fa6300b57b0 msgr2=0x7fa6300b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:03.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 -- 172.21.15.114:0/2369926371 shutdown_connections 2023-11-24T06:25:03.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.075+0000 7fa6353b1700 1 -- 172.21.15.114:0/2369926371 wait complete. 2023-11-24T06:25:03.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 Processor -- start 2023-11-24T06:25:03.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 -- start start 2023-11-24T06:25:03.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300b2870 0x7fa6301f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa63006de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63015cb00 con 0x7fa6300b04b0 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63015cde0 con 0x7fa6300b2870 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.077+0000 7fa6353b1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63015d0c0 con 0x7fa63015c160 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39268/0 (socket says 172.21.15.114:39268) 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 -- 172.21.15.114:0/3873804993 learned_addr learned my addr 172.21.15.114:0/3873804993 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 -- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 msgr2=0x7fa63006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa63006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 -- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300b2870 msgr2=0x7fa6301f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300b2870 0x7fa6301f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 -- 172.21.15.114:0/3873804993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009a70 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa62effd700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e850 tx=0x7fa62400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240095b0 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62400ee50 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63015d3a0 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62400ca70 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.078+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa630148b90 con 0x7fa6300b04b0 2023-11-24T06:25:03.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.079+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa630067f60 con 0x7fa6300b04b0 2023-11-24T06:25:03.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.080+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa62400cbd0 con 0x7fa6300b04b0 2023-11-24T06:25:03.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.080+0000 7fa6177fe700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 0x7fa61c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.080+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa6240895a0 con 0x7fa6300b04b0 2023-11-24T06:25:03.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.080+0000 7fa62e7fc700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 0x7fa61c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.080+0000 7fa62e7fc700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 0x7fa61c06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6200011e0 tx=0x7fa6200079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.084+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa624054660 con 0x7fa6300b04b0 2023-11-24T06:25:03.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.290+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa63006d930 con 0x7fa6300b04b0 2023-11-24T06:25:03.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.290+0000 7fa6177fe700 1 -- 172.21.15.114:0/3873804993 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa624057c80 con 0x7fa6300b04b0 2023-11-24T06:25:03.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 msgr2=0x7fa61c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 0x7fa61c06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6200011e0 tx=0x7fa6200079b0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 msgr2=0x7fa630148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e850 tx=0x7fa62400ec10 comp rx=0 tx=0).stop 2023-11-24T06:25:03.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 shutdown_connections 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa61c06aed0 0x7fa61c06d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa63015c160 0x7fa63006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6300b04b0 0x7fa630148650 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 --2- 172.21.15.114:0/3873804993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6300b2870 0x7fa6301f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.292+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 >> 172.21.15.114:0/3873804993 conn(0x7fa6300b57b0 msgr2=0x7fa6300b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.293+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 shutdown_connections 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.293+0000 7fa6353b1700 1 -- 172.21.15.114:0/3873804993 wait complete. 2023-11-24T06:25:03.296 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:25:03.307 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-24T06:25:03.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:03.307 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-24T06:25:03.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:25:03.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:03.356 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:25:03.376 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:25:03.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:03.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.494+0000 7fb7ab576700 1 Processor -- start 2023-11-24T06:25:03.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 -- start start 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb7a415ddd0 0x7fb7a40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 0x7fb7a40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40b1d40 con 0x7fb7a40d1b30 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40b28c0 con 0x7fb7a415ddd0 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.495+0000 7fb7ab576700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a406c000 con 0x7fb7a415e1b0 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb7a415ddd0 0x7fb7a40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb7a415ddd0 0x7fb7a40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39276/0 (socket says 172.21.15.114:39276) 2023-11-24T06:25:03.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 -- 172.21.15.114:0/1180280431 learned_addr learned my addr 172.21.15.114:0/1180280431 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 -- 172.21.15.114:0/1180280431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 msgr2=0x7fb7a40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a8b11700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 0x7fb7a40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 0x7fb7a40cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 -- 172.21.15.114:0/1180280431 >> v1:172.21.15.172:6789/0 conn(0x7fb7a40d1b30 legacy=0x7fb7a4161c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 -- 172.21.15.114:0/1180280431 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40b1f90 con 0x7fb7a415ddd0 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a8b11700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 0x7fb7a40cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb7a9312700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415ddd0 0x7fb7a40b1680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098f0 tx=0x7fb794009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d69381db843fd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb79bfff700 1 -- 172.21.15.114:0/1180280431 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794004230 con 0x7fb7a415ddd0 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb79bfff700 1 -- 172.21.15.114:0/1180280431 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794004390 con 0x7fb7a415ddd0 2023-11-24T06:25:03.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.496+0000 7fb79bfff700 1 -- 172.21.15.114:0/1180280431 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940036a0 con 0x7fb7a415ddd0 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 -- 172.21.15.114:0/1180280431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415ddd0 msgr2=0x7fb7a40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415ddd0 0x7fb7a40b1680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098f0 tx=0x7fb794009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 -- 172.21.15.114:0/1180280431 shutdown_connections 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415ddd0 0x7fb7a40b1680 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 --2- 172.21.15.114:0/1180280431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415e1b0 0x7fb7a40cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 -- 172.21.15.114:0/1180280431 >> 172.21.15.114:0/1180280431 conn(0x7fb7a40731a0 msgr2=0x7fb7a409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 -- 172.21.15.114:0/1180280431 shutdown_connections 2023-11-24T06:25:03.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.497+0000 7fb7ab576700 1 -- 172.21.15.114:0/1180280431 wait complete. 2023-11-24T06:25:03.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.498+0000 7fb7ab576700 1 Processor -- start 2023-11-24T06:25:03.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.498+0000 7fb7ab576700 1 -- start start 2023-11-24T06:25:03.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7a40d1b30 0x7fb7a41f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415ddd0 0x7fb7a415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4158d20 con 0x7fb7a415e1b0 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4155850 con 0x7fb7a40d1b30 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4158a40 con 0x7fb7a415ddd0 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39284/0 (socket says 172.21.15.114:39284) 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 -- 172.21.15.114:0/3796263984 learned_addr learned my addr 172.21.15.114:0/3796263984 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a8b11700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415ddd0 0x7fb7a415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9312700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7a40d1b30 0x7fb7a41f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 -- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415ddd0 msgr2=0x7fb7a415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415ddd0 0x7fb7a415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 -- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7a40d1b30 msgr2=0x7fb7a41f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7a40d1b30 0x7fb7a41f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 -- 172.21.15.114:0/3796263984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009550 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7a9b13700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b780 tx=0x7fb79c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00f840 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c00f9a0 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40b2ed0 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.499+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00fd50 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.500+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a4155b10 con 0x7fb7a415e1b0 2023-11-24T06:25:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.500+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4067f60 con 0x7fb7a415e1b0 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.501+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb79c00d3d0 con 0x7fb7a415e1b0 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.501+0000 7fb799ffb700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 0x7fb78806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.501+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb79c0899e0 con 0x7fb7a415e1b0 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.501+0000 7fb7a9312700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 0x7fb78806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.502+0000 7fb7a9312700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 0x7fb78806d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4155620 tx=0x7fb794008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.505+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb79c054b50 con 0x7fb7a415e1b0 2023-11-24T06:25:03.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.737+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7a40b7c60 con 0x7fb7a415e1b0 2023-11-24T06:25:03.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.738+0000 7fb799ffb700 1 -- 172.21.15.114:0/3796263984 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fb79c01c090 con 0x7fb7a415e1b0 2023-11-24T06:25:03.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:03.741 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 msgr2=0x7fb78806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 0x7fb78806d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4155620 tx=0x7fb794008030 comp rx=0 tx=0).stop 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 msgr2=0x7fb7a4157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b780 tx=0x7fb79c00bb40 comp rx=0 tx=0).stop 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 shutdown_connections 2023-11-24T06:25:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb78806aed0 0x7fb78806d390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a415e1b0 0x7fb7a4157f40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7a40d1b30 0x7fb7a41f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 --2- 172.21.15.114:0/3796263984 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a415ddd0 0x7fb7a415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 >> 172.21.15.114:0/3796263984 conn(0x7fb7a40731a0 msgr2=0x7fb7a40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 shutdown_connections 2023-11-24T06:25:03.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.740+0000 7fb7ab576700 1 -- 172.21.15.114:0/3796263984 wait complete. 2023-11-24T06:25:03.745 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:25:03.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.873+0000 7fe5f4950700 1 Processor -- start 2023-11-24T06:25:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.873+0000 7fe5f4950700 1 -- start start 2023-11-24T06:25:03.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5f4950700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe5f015e080 0x7fe5f00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5f4950700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5f006c710 0x7fe5f00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5f4950700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f00ae430 con 0x7fe5f00af4d0 2023-11-24T06:25:03.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5f4950700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00af0b0 con 0x7fe5f006c710 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5f4950700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00b05a0 con 0x7fe5f015e080 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5f006c710 0x7fe5f00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5f006c710 0x7fe5f00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39292/0 (socket says 172.21.15.114:39292) 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 -- 172.21.15.114:0/4091034737 learned_addr learned my addr 172.21.15.114:0/4091034737 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5ee59c700 1 --2- 172.21.15.114:0/4091034737 >> v2:172.21.15.172:3300/0 conn(0x7fe5f015e080 0x7fe5f00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 -- 172.21.15.114:0/4091034737 >> v2:172.21.15.172:3300/0 conn(0x7fe5f015e080 msgr2=0x7fe5f00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 --2- 172.21.15.114:0/4091034737 >> v2:172.21.15.172:3300/0 conn(0x7fe5f015e080 0x7fe5f00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 -- 172.21.15.114:0/4091034737 >> v1:172.21.15.172:6789/0 conn(0x7fe5f00af4d0 legacy=0x7fe5f006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 -- 172.21.15.114:0/4091034737 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00ae710 con 0x7fe5f006c710 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5eed9d700 1 --2- 172.21.15.114:0/4091034737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f006c710 0x7fe5f00afb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0072a90 tx=0x7fe5e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca0e4d91ba0e62c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5ed59a700 1 -- 172.21.15.114:0/4091034737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000b860 con 0x7fe5f006c710 2023-11-24T06:25:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5ed59a700 1 -- 172.21.15.114:0/4091034737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e000b9c0 con 0x7fe5f006c710 2023-11-24T06:25:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.874+0000 7fe5ed59a700 1 -- 172.21.15.114:0/4091034737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000bd50 con 0x7fe5f006c710 2023-11-24T06:25:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 -- 172.21.15.114:0/4091034737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f006c710 msgr2=0x7fe5f00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 --2- 172.21.15.114:0/4091034737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f006c710 0x7fe5f00afb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0072a90 tx=0x7fe5e0009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 -- 172.21.15.114:0/4091034737 shutdown_connections 2023-11-24T06:25:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 --2- 172.21.15.114:0/4091034737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f006c710 0x7fe5f00afb90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 --2- 172.21.15.114:0/4091034737 >> v2:172.21.15.172:3300/0 conn(0x7fe5f015e080 0x7fe5f00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 -- 172.21.15.114:0/4091034737 >> 172.21.15.114:0/4091034737 conn(0x7fe5f0071b90 msgr2=0x7fe5f0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 -- 172.21.15.114:0/4091034737 shutdown_connections 2023-11-24T06:25:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.875+0000 7fe5f4950700 1 -- 172.21.15.114:0/4091034737 wait complete. 2023-11-24T06:25:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.876+0000 7fe5f4950700 1 Processor -- start 2023-11-24T06:25:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.876+0000 7fe5f4950700 1 -- start start 2023-11-24T06:25:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f006c710 0x7fe5f0141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f015e080 0x7fe5f0160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f015a8f0 con 0x7fe5f00af4d0 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0141fb0 con 0x7fe5f006c710 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5f4950700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f015ab40 con 0x7fe5f015e080 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39296/0 (socket says 172.21.15.114:39296) 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 -- 172.21.15.114:0/1062347610 learned_addr learned my addr 172.21.15.114:0/1062347610 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5ee59c700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f006c710 0x7fe5f0141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5eed9d700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f015e080 0x7fe5f0160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 -- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f015e080 msgr2=0x7fe5f0160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f015e080 0x7fe5f0160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 -- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f006c710 msgr2=0x7fe5f0141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f006c710 0x7fe5f0141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.877+0000 7fe5edd9b700 1 -- 172.21.15.114:0/1062347610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009550 con 0x7fe5f00af4d0 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5edd9b700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c4b0 tx=0x7fe5d800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80052d0 con 0x7fe5f00af4d0 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8005430 con 0x7fe5f00af4d0 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80038c0 con 0x7fe5f00af4d0 2023-11-24T06:25:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f015ae20 con 0x7fe5f00af4d0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.878+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f015b3e0 con 0x7fe5f00af4d0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.879+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f00753f0 con 0x7fe5f00af4d0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.879+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5d8003a20 con 0x7fe5f00af4d0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.880+0000 7fe5e6ffd700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.880+0000 7fe5ee59c700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.880+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe5d8088950 con 0x7fe5f00af4d0 2023-11-24T06:25:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.880+0000 7fe5ee59c700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5f01f1480 tx=0x7fe5e8009800 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:03.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:03.884+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5d8053a10 con 0x7fe5f00af4d0 2023-11-24T06:25:04.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.080+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5f0067f60 con 0x7fe5f00af4d0 2023-11-24T06:25:04.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.081+0000 7fe5e6ffd700 1 -- 172.21.15.114:0/1062347610 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fe5d8017090 con 0x7fe5f00af4d0 2023-11-24T06:25:04.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:04.084 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:59.029214+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 msgr2=0x7fe5d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5f01f1480 tx=0x7fe5e8009800 comp rx=0 tx=0).stop 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 msgr2=0x7fe5f0160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c4b0 tx=0x7fe5d800c7c0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 shutdown_connections 2023-11-24T06:25:04.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f015e080 0x7fe5f0160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f006c710 0x7fe5f0141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 --2- 172.21.15.114:0/1062347610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5f00af4d0 0x7fe5f0160ab0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 >> 172.21.15.114:0/1062347610 conn(0x7fe5f0071b90 msgr2=0x7fe5f00b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 shutdown_connections 2023-11-24T06:25:04.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.083+0000 7fe5f4950700 1 -- 172.21.15.114:0/1062347610 wait complete. 2023-11-24T06:25:04.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:24:17.780078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:24:18.203282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:04.099 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:04.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.220+0000 7f0647d86700 1 Processor -- start 2023-11-24T06:25:04.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.220+0000 7f0647d86700 1 -- start start 2023-11-24T06:25:04.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0647d86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f064006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0647d86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06400b4370 0x7f064015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0647d86700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064006b530 con 0x7f064015b370 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0647d86700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06400ae5e0 con 0x7f06400b4370 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0647d86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064015be30 con 0x7f06401587f0 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0646323700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06400b4370 0x7f064015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0645321700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f064015b370 0x7f06400b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55010/0 (socket says 172.21.15.114:55010) 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0645321700 1 -- 172.21.15.114:0/1014793943 learned_addr learned my addr 172.21.15.114:0/1014793943 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0645b22700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f064006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0646323700 1 -- 172.21.15.114:0/1014793943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 msgr2=0x7f064006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:04.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0646323700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f064006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0646323700 1 -- 172.21.15.114:0/1014793943 >> v1:172.21.15.114:6789/0 conn(0x7f064015b370 legacy=0x7f06400b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0646323700 1 -- 172.21.15.114:0/1014793943 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064015c110 con 0x7f06400b4370 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.221+0000 7f0644b20700 1 -- 172.21.15.114:0/1014793943 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377810907 0 0) 0x7f064006b530 con 0x7f064015b370 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0646323700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b4370 0x7f064015b7f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c29c8ffa5f08f8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0644b20700 1 -- 172.21.15.114:0/1014793943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800e070 con 0x7f06400b4370 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0644b20700 1 -- 172.21.15.114:0/1014793943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063800bbd0 con 0x7f06400b4370 2023-11-24T06:25:04.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0644b20700 1 -- 172.21.15.114:0/1014793943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638012430 con 0x7f06400b4370 2023-11-24T06:25:04.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 -- 172.21.15.114:0/1014793943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b4370 msgr2=0x7f064015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b4370 0x7f064015b7f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:04.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 -- 172.21.15.114:0/1014793943 shutdown_connections 2023-11-24T06:25:04.228 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f064006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 --2- 172.21.15.114:0/1014793943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400b4370 0x7f064015b7f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.229 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 -- 172.21.15.114:0/1014793943 >> 172.21.15.114:0/1014793943 conn(0x7f06400d0bf0 msgr2=0x7f06400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 -- 172.21.15.114:0/1014793943 shutdown_connections 2023-11-24T06:25:04.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.222+0000 7f0647d86700 1 -- 172.21.15.114:0/1014793943 wait complete. 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.224+0000 7f0647d86700 1 Processor -- start 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.224+0000 7f0647d86700 1 -- start start 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06400b4370 0x7f0640147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f0640145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640147bc0 con 0x7f064015b370 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401465e0 con 0x7f06400b4370 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0647d86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06401468c0 con 0x7f06401587f0 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39312/0 (socket says 172.21.15.114:39312) 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 -- 172.21.15.114:0/3663671455 learned_addr learned my addr 172.21.15.114:0/3663671455 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:04.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0645b22700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06400b4370 0x7f0640147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0645321700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f0640145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 -- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 msgr2=0x7f0640145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f0640145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 -- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06400b4370 msgr2=0x7f0640147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06400b4370 0x7f0640147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 -- 172.21.15.114:0/3663671455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009a70 con 0x7f064015b370 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.225+0000 7f0646323700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0638013000 tx=0x7f0638004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.226+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638020070 con 0x7f064015b370 2023-11-24T06:25:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.226+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06380098f0 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.226+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0640146ba0 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.226+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638012430 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.226+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0640147160 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.227+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06401472f0 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.227+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0638024070 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.227+0000 7f06367fc700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 0x7f062406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.227+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f063808c8d0 con 0x7f064015b370 2023-11-24T06:25:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.227+0000 7f0645b22700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 0x7f062406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.228+0000 7f0645b22700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 0x7f062406d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0630009930 tx=0x7f0630008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.237 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.232+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0638057ab0 con 0x7f064015b370 2023-11-24T06:25:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.420+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0640068fb0 con 0x7f064015b370 2023-11-24T06:25:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.420+0000 7f06367fc700 1 -- 172.21.15.114:0/3663671455 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f063802b020 con 0x7f064015b370 2023-11-24T06:25:04.423 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 msgr2=0x7f062406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 0x7f062406d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0630009930 tx=0x7f0630008040 comp rx=0 tx=0).stop 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 msgr2=0x7f0640145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0638013000 tx=0x7f0638004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 shutdown_connections 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f062406aed0 0x7f062406d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f064015b370 0x7f0640145420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06400b4370 0x7f0640147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 --2- 172.21.15.114:0/3663671455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06401587f0 0x7f0640145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 >> 172.21.15.114:0/3663671455 conn(0x7f06400d0bf0 msgr2=0x7f064006ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 shutdown_connections 2023-11-24T06:25:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.422+0000 7f0647d86700 1 -- 172.21.15.114:0/3663671455 wait complete. 2023-11-24T06:25:04.437 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:04.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.557+0000 7f5db921a700 1 Processor -- start 2023-11-24T06:25:04.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.557+0000 7f5db921a700 1 -- start start 2023-11-24T06:25:04.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db921a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db415bf50 0x7f5db40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db921a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5db40c6c30 0x7f5db40cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db921a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40d0140 con 0x7f5db40ce830 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db921a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db415c330 con 0x7f5db40c6c30 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db921a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cf3c0 con 0x7f5db415bf50 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db359e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5db40c6c30 0x7f5db40cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db259c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5db40ce830 0x7f5db40cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55024/0 (socket says 172.21.15.114:55024) 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.558+0000 7f5db259c700 1 -- 172.21.15.114:0/3887976084 learned_addr learned my addr 172.21.15.114:0/3887976084 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db2d9d700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db415bf50 0x7f5db40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db359e700 1 -- 172.21.15.114:0/3887976084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db415bf50 msgr2=0x7f5db40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db359e700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db415bf50 0x7f5db40ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db359e700 1 -- 172.21.15.114:0/3887976084 >> v1:172.21.15.114:6789/0 conn(0x7f5db40ce830 legacy=0x7f5db40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db1d9b700 1 -- 172.21.15.114:0/3887976084 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142092977 0 0) 0x7f5db40d0140 con 0x7f5db40ce830 2023-11-24T06:25:04.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db359e700 1 -- 172.21.15.114:0/3887976084 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40d03e0 con 0x7f5db40c6c30 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db359e700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40cc6c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009d90 tx=0x7f5dac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a5cfbf074033cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db1d9b700 1 -- 172.21.15.114:0/3887976084 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00e070 con 0x7f5db40c6c30 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db1d9b700 1 -- 172.21.15.114:0/3887976084 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac00bbd0 con 0x7f5db40c6c30 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db1d9b700 1 -- 172.21.15.114:0/3887976084 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac012430 con 0x7f5db40c6c30 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.559+0000 7f5db921a700 1 -- 172.21.15.114:0/3887976084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 msgr2=0x7f5db40cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40cc6c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009d90 tx=0x7f5dac00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 -- 172.21.15.114:0/3887976084 shutdown_connections 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db415bf50 0x7f5db40ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 --2- 172.21.15.114:0/3887976084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40cc6c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 -- 172.21.15.114:0/3887976084 >> 172.21.15.114:0/3887976084 conn(0x7f5db40b3780 msgr2=0x7f5db40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 -- 172.21.15.114:0/3887976084 shutdown_connections 2023-11-24T06:25:04.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.560+0000 7f5db921a700 1 -- 172.21.15.114:0/3887976084 wait complete. 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.561+0000 7f5db921a700 1 Processor -- start 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.561+0000 7f5db921a700 1 -- start start 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db40ce830 0x7f5db4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db415bf50 0x7f5db40b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b56b0 con 0x7f5db40c6c30 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b5990 con 0x7f5db415bf50 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db921a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b5c70 con 0x7f5db40ce830 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39322/0 (socket says 172.21.15.114:39322) 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 -- 172.21.15.114:0/747681471 learned_addr learned my addr 172.21.15.114:0/747681471 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:04.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db259c700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db40ce830 0x7f5db4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db359e700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db415bf50 0x7f5db40b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 -- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db40ce830 msgr2=0x7f5db4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db40ce830 0x7f5db4155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 -- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db415bf50 msgr2=0x7f5db40b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db415bf50 0x7f5db40b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 -- 172.21.15.114:0/747681471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac009a70 con 0x7f5db40c6c30 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5db2d9d700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5da800d800 tx=0x7f5da800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.562+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80125f0 con 0x7f5db40c6c30 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.563+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8012750 con 0x7f5db40c6c30 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.563+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8012b00 con 0x7f5db40c6c30 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.563+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db40b5f50 con 0x7f5db40c6c30 2023-11-24T06:25:04.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.563+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db40b2870 con 0x7f5db40c6c30 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.563+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db40b2a00 con 0x7f5db40c6c30 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.564+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5da8012c60 con 0x7f5db40c6c30 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.564+0000 7f5d9b7fe700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 0x7f5da006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.564+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5da8089a10 con 0x7f5db40c6c30 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.564+0000 7f5db259c700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 0x7f5da006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.565+0000 7f5db259c700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 0x7f5da006d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5db415acd0 tx=0x7f5da4007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.569+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5db40b2a00 con 0x7f5db40c6c30 2023-11-24T06:25:04.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.759+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5db40b2a00 con 0x7f5db40c6c30 2023-11-24T06:25:04.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.759+0000 7f5d9b7fe700 1 -- 172.21.15.114:0/747681471 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5db40b2a00 con 0x7f5db40c6c30 2023-11-24T06:25:04.762 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:04.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 msgr2=0x7f5da006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 0x7f5da006d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5db415acd0 tx=0x7f5da4007be0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 msgr2=0x7f5db40b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5da800d800 tx=0x7f5da800dbc0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 shutdown_connections 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5da006aed0 0x7f5da006d390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db415bf50 0x7f5db40b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5db40c6c30 0x7f5db40b2330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 --2- 172.21.15.114:0/747681471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5db40ce830 0x7f5db4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.761+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 >> 172.21.15.114:0/747681471 conn(0x7f5db40b3780 msgr2=0x7f5db40c7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.762+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 shutdown_connections 2023-11-24T06:25:04.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.762+0000 7f5db921a700 1 -- 172.21.15.114:0/747681471 wait complete. 2023-11-24T06:25:04.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:04.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 Processor -- start 2023-11-24T06:25:04.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 -- start start 2023-11-24T06:25:04.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f76f40b5da0 0x7f76f40afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f40b3c60 con 0x7f76f40746a0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f40b58c0 con 0x7f76f415e160 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.897+0000 7f76f9cd9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40b4500 con 0x7f76f40b5da0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f2ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f76f40746a0 0x7f76f406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55040/0 (socket says 172.21.15.114:55040) 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f2ffd700 1 -- 172.21.15.114:0/1422797206 learned_addr learned my addr 172.21.15.114:0/1422797206 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f3fff700 1 --2- 172.21.15.114:0/1422797206 >> v2:172.21.15.172:3300/0 conn(0x7f76f40b5da0 0x7f76f40afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511185099 0 0) 0x7f76f40b3c60 con 0x7f76f40746a0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e00035b0 con 0x7f76f40746a0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074910298 0 0) 0x7f76f40b58c0 con 0x7f76f415e160 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76f40b3c60 con 0x7f76f415e160 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594328305 0 0) 0x7f76e00035b0 con 0x7f76f40746a0 2023-11-24T06:25:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f3fff700 1 -- 172.21.15.114:0/1422797206 >> v1:172.21.15.172:6789/0 conn(0x7f76f415e160 legacy=0x7f76f4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f3fff700 1 -- 172.21.15.114:0/1422797206 >> v1:172.21.15.114:6789/0 conn(0x7f76f40746a0 legacy=0x7f76f406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f3fff700 1 -- 172.21.15.114:0/1422797206 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f406bdc0 con 0x7f76f40b5da0 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f3fff700 1 --2- 172.21.15.114:0/1422797206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f40b5da0 0x7f76f40afc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009840 tx=0x7f76ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc368ea8f2c919d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc001670 con 0x7f76f40b5da0 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.898+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec004030 con 0x7f76f40b5da0 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f27fc700 1 -- 172.21.15.114:0/1422797206 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0043a0 con 0x7f76f40b5da0 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1422797206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f40b5da0 msgr2=0x7f76f40afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1422797206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f40b5da0 0x7f76f40afc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009840 tx=0x7f76ec009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1422797206 shutdown_connections 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1422797206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f40b5da0 0x7f76f40afc90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1422797206 >> 172.21.15.114:0/1422797206 conn(0x7f76f40d0bf0 msgr2=0x7f76f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1422797206 shutdown_connections 2023-11-24T06:25:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.899+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1422797206 wait complete. 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.900+0000 7f76f9cd9700 1 Processor -- start 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.900+0000 7f76f9cd9700 1 -- start start 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76f40b5da0 0x7f76f4141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f415e160 0x7f76f41419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4160a10 con 0x7f76f40746a0 2023-11-24T06:25:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4160cf0 con 0x7f76f415e160 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f9cd9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4160fd0 con 0x7f76f40b5da0 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39336/0 (socket says 172.21.15.114:39336) 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f37fe700 1 -- 172.21.15.114:0/1983723495 learned_addr learned my addr 172.21.15.114:0/1983723495 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f3fff700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f415e160 0x7f76f41419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.901+0000 7f76f2ffd700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76f40b5da0 0x7f76f4141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 -- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76f40b5da0 msgr2=0x7f76f4141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76f40b5da0 0x7f76f4141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 -- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f415e160 msgr2=0x7f76f41419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f415e160 0x7f76f41419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 -- 172.21.15.114:0/1983723495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0094a0 con 0x7f76f40746a0 2023-11-24T06:25:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f37fe700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76e400cbe0 tx=0x7f76e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003c00 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4003d80 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f41612b0 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400abc0 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.902+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f415b6e0 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.903+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4161620 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.903+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76e400ad20 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.904+0000 7f76ebfff700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 0x7f76d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.904+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f76e4089730 con 0x7f76f40746a0 2023-11-24T06:25:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.904+0000 7f76f2ffd700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 0x7f76d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:04.909 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.904+0000 7f76f2ffd700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 0x7f76d406d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76f4152810 tx=0x7f76dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:04.908+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76f4161620 con 0x7f76f40746a0 2023-11-24T06:25:05.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.101+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76f4161620 con 0x7f76f40746a0 2023-11-24T06:25:05.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.101+0000 7f76ebfff700 1 -- 172.21.15.114:0/1983723495 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76f4161620 con 0x7f76f40746a0 2023-11-24T06:25:05.104 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.103+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 msgr2=0x7f76d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.103+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 0x7f76d406d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f76f4152810 tx=0x7f76dc009040 comp rx=0 tx=0).stop 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.103+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 msgr2=0x7f76f415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.103+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f76e400cbe0 tx=0x7f76e400a060 comp rx=0 tx=0).stop 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 shutdown_connections 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f76d406aed0 0x7f76d406d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76f415e160 0x7f76f41419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76f40746a0 0x7f76f415afc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 --2- 172.21.15.114:0/1983723495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76f40b5da0 0x7f76f4141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 >> 172.21.15.114:0/1983723495 conn(0x7f76f40d0bf0 msgr2=0x7f76f415d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 shutdown_connections 2023-11-24T06:25:05.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.104+0000 7f76f9cd9700 1 -- 172.21.15.114:0/1983723495 wait complete. 2023-11-24T06:25:05.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:05.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.236+0000 7fd9022f1700 1 Processor -- start 2023-11-24T06:25:05.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.236+0000 7fd9022f1700 1 -- start start 2023-11-24T06:25:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd9022f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8fc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd9022f1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0ae430 con 0x7fd8fc15e080 2023-11-24T06:25:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd9022f1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0ae710 con 0x7fd8fc06c710 2023-11-24T06:25:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd9022f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0af0b0 con 0x7fd8fc0af4d0 2023-11-24T06:25:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8fbfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd8fc15e080 0x7fd8fc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55052/0 (socket says 172.21.15.114:55052) 2023-11-24T06:25:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8fbfff700 1 -- 172.21.15.114:0/577568355 learned_addr learned my addr 172.21.15.114:0/577568355 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd90088e700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8fc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194461920 0 0) 0x7fd8fc0ae430 con 0x7fd8fc15e080 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e40035b0 con 0x7fd8fc15e080 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788239235 0 0) 0x7fd8e40035b0 con 0x7fd8fc15e080 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 msgr2=0x7fd8fc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8fc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 >> v1:172.21.15.172:6789/0 conn(0x7fd8fc06c710 legacy=0x7fd8fc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0ae9f0 con 0x7fd8fc15e080 2023-11-24T06:25:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fd8ec001670 con 0x7fd8fc15e080 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 >> v1:172.21.15.114:6789/0 conn(0x7fd8fc15e080 legacy=0x7fd8fc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e4003a30 0x7fd8e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8e4004d90 0x7fd8e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0af0b0 con 0x7fd8e4004d90 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400bac0 con 0x7fd8e4003a30 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.237+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400bc80 con 0x7fd8fc0af4d0 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8e4004d90 0x7fd8e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8fbfff700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e4003a30 0x7fd8e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8fb7fe700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 -- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 msgr2=0x7fd8e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 -- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e4003a30 msgr2=0x7fd8e4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e4003a30 0x7fd8e4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 -- 172.21.15.114:0/577568355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e40035b0 con 0x7fd8e4004d90 2023-11-24T06:25:05.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8fb7fe700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0af4d0 0x7fd8e4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:05.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd90088e700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8e4004d90 0x7fd8e4005210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009f90 tx=0x7fd8e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73efa07522665d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400ec30 con 0x7fd8e4004d90 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f400f040 con 0x7fd8e4004d90 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.238+0000 7fd8faffd700 1 -- 172.21.15.114:0/577568355 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4014480 con 0x7fd8e4004d90 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 -- 172.21.15.114:0/577568355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8e4004d90 msgr2=0x7fd8e4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 --2- 172.21.15.114:0/577568355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8e4004d90 0x7fd8e4005210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009f90 tx=0x7fd8e4005880 comp rx=0 tx=0).stop 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd90088e700 1 -- 172.21.15.114:0/577568355 reap_dead start 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 -- 172.21.15.114:0/577568355 shutdown_connections 2023-11-24T06:25:05.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 -- 172.21.15.114:0/577568355 >> 172.21.15.114:0/577568355 conn(0x7fd8fc071b90 msgr2=0x7fd8fc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 -- 172.21.15.114:0/577568355 shutdown_connections 2023-11-24T06:25:05.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.239+0000 7fd9022f1700 1 -- 172.21.15.114:0/577568355 wait complete. 2023-11-24T06:25:05.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.240+0000 7fd9022f1700 1 Processor -- start 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.240+0000 7fd9022f1700 1 -- start start 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8fc1f2e50 0x7fd8fc1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 0x7fd8fc0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b2090 con 0x7fd8fc1f3230 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc1f36b0 con 0x7fd8fc1f2e50 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc1548c0 con 0x7fd8fc0b16c0 2023-11-24T06:25:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39358/0 (socket says 172.21.15.114:39358) 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 -- 172.21.15.114:0/411036371 learned_addr learned my addr 172.21.15.114:0/411036371 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fbfff700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8fc1f2e50 0x7fd8fc1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd90088e700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 0x7fd8fc0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 -- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 msgr2=0x7fd8fc0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 0x7fd8fc0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 -- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8fc1f2e50 msgr2=0x7fd8fc1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8fc1f2e50 0x7fd8fc1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 -- 172.21.15.114:0/411036371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc153f90 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd90088e700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 0x7fd8fc0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8fb7fe700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000e7b0 tx=0x7fd8f000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0001330 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00052e0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc154270 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.241+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0003680 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.242+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc1499d0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.243+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc0753f0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.243+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8f00037e0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.243+0000 7fd8f8ff9700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 0x7fd8e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.243+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd8f0089290 con 0x7fd8fc1f3230 2023-11-24T06:25:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.243+0000 7fd8fbfff700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 0x7fd8e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.244+0000 7fd8fbfff700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 0x7fd8e806d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0017a0 tx=0x7fd8ec001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.248+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8f00542c0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.480+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8fc067f60 con 0x7fd8fc1f3230 2023-11-24T06:25:05.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.481+0000 7fd8f8ff9700 1 -- 172.21.15.114:0/411036371 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fd8f00578e0 con 0x7fd8fc1f3230 2023-11-24T06:25:05.483 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:05.484 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:25:05.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 msgr2=0x7fd8e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 0x7fd8e806d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0017a0 tx=0x7fd8ec001550 comp rx=0 tx=0).stop 2023-11-24T06:25:05.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 msgr2=0x7fd8fc160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000e7b0 tx=0x7fd8f000eb70 comp rx=0 tx=0).stop 2023-11-24T06:25:05.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 shutdown_connections 2023-11-24T06:25:05.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd8e806aed0 0x7fd8e806d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8fc0b16c0 0x7fd8fc0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8fc1f2e50 0x7fd8fc1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 --2- 172.21.15.114:0/411036371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8fc1f3230 0x7fd8fc160db0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 >> 172.21.15.114:0/411036371 conn(0x7fd8fc071b90 msgr2=0x7fd8fc06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 shutdown_connections 2023-11-24T06:25:05.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.483+0000 7fd9022f1700 1 -- 172.21.15.114:0/411036371 wait complete. 2023-11-24T06:25:05.489 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:25:05.498 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.616+0000 7f39baf89700 1 Processor -- start 2023-11-24T06:25:05.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.616+0000 7f39baf89700 1 -- start start 2023-11-24T06:25:05.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39baf89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39b40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39baf89700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f39b406c710 0x7f39b40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39baf89700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40ae430 con 0x7f39b40af4d0 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39baf89700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40af0b0 con 0x7f39b406c710 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39baf89700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40b05a0 con 0x7f39b415e080 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b3fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f39b40af4d0 0x7f39b406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55074/0 (socket says 172.21.15.114:55074) 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b3fff700 1 -- 172.21.15.114:0/3383723162 learned_addr learned my addr 172.21.15.114:0/3383723162 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39b40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b9526700 1 --2- 172.21.15.114:0/3383723162 >> v2:172.21.15.172:3300/0 conn(0x7f39b406c710 0x7f39b40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045415052 0 0) 0x7f39b40ae430 con 0x7f39b40af4d0 2023-11-24T06:25:05.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a40035b0 con 0x7f39b40af4d0 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122003162 0 0) 0x7f39a40035b0 con 0x7f39b40af4d0 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 msgr2=0x7f39b40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39b40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 >> v2:172.21.15.172:3300/0 conn(0x7f39b406c710 msgr2=0x7f39b40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 --2- 172.21.15.114:0/3383723162 >> v2:172.21.15.172:3300/0 conn(0x7f39b406c710 0x7f39b40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40ae710 con 0x7f39b40af4d0 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f39a0001670 con 0x7f39b40af4d0 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39b40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 >> v1:172.21.15.114:6789/0 conn(0x7f39b40af4d0 legacy=0x7f39b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4003a30 0x7f39a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a4004850 0x7f39a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40af0b0 con 0x7f39a4004850 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40b05a0 con 0x7f39a4003a30 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.617+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a400ba10 con 0x7f39b415e080 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a4004850 0x7f39a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b9526700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4003a30 0x7f39a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 -- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 msgr2=0x7f39a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415e080 0x7f39a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 -- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4003a30 msgr2=0x7f39a4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4003a30 0x7f39a4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 -- 172.21.15.114:0/3383723162 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80094a0 con 0x7f39a4004850 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a4004850 0x7f39a4004cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c210 tx=0x7f39a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e494f3dc4be02a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0007c10 con 0x7f39a4004850 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a000cdd0 con 0x7f39a4004850 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.618+0000 7f39b37fe700 1 -- 172.21.15.114:0/3383723162 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0005ba0 con 0x7f39a4004850 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 -- 172.21.15.114:0/3383723162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a4004850 msgr2=0x7f39a4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 --2- 172.21.15.114:0/3383723162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a4004850 0x7f39a4004cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c210 tx=0x7f39a4005880 comp rx=0 tx=0).stop 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39b9526700 1 -- 172.21.15.114:0/3383723162 reap_dead start 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 -- 172.21.15.114:0/3383723162 shutdown_connections 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 -- 172.21.15.114:0/3383723162 >> 172.21.15.114:0/3383723162 conn(0x7f39b4071b90 msgr2=0x7f39b4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 -- 172.21.15.114:0/3383723162 shutdown_connections 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.619+0000 7f39baf89700 1 -- 172.21.15.114:0/3383723162 wait complete. 2023-11-24T06:25:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.620+0000 7f39baf89700 1 Processor -- start 2023-11-24T06:25:05.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.620+0000 7f39baf89700 1 -- start start 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4004850 0x7f39b4152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 0x7f39b415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4158800 con 0x7f39b41416e0 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b415baa0 con 0x7f39a4004850 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4158da0 con 0x7f39b415a840 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39378/0 (socket says 172.21.15.114:39378) 2023-11-24T06:25:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 -- 172.21.15.114:0/3715464685 learned_addr learned my addr 172.21.15.114:0/3715464685 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4004850 0x7f39b4152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b9526700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 0x7f39b415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 -- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 msgr2=0x7f39b415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 0x7f39b415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 -- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4004850 msgr2=0x7f39b4152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4004850 0x7f39b4152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 -- 172.21.15.114:0/3715464685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b41420a0 con 0x7f39b41416e0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b9526700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 0x7f39b415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b3fff700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39a0000f50 tx=0x7f39a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0001630 con 0x7f39b41416e0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a00070d0 con 0x7f39b41416e0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b415b230 con 0x7f39b41416e0 2023-11-24T06:25:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.621+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0007480 con 0x7f39b41416e0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.622+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b415b7c0 con 0x7f39b41416e0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.622+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40753f0 con 0x7f39b41416e0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.623+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39a0019030 con 0x7f39b41416e0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.623+0000 7f39b17fa700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 0x7f399806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.623+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f39a008aba0 con 0x7f39b41416e0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.623+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 0x7f399806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.624+0000 7f39b8d25700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 0x7f399806d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009470 tx=0x7f39a8009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.628+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39a0055c60 con 0x7f39b41416e0 2023-11-24T06:25:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.818+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b4067f60 con 0x7f39b41416e0 2023-11-24T06:25:05.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.819+0000 7f39b17fa700 1 -- 172.21.15.114:0/3715464685 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f39a0017090 con 0x7f39b41416e0 2023-11-24T06:25:05.822 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:05.822 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:59.029214+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.821+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 msgr2=0x7f399806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.821+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 0x7f399806d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009470 tx=0x7f39a8009210 comp rx=0 tx=0).stop 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.821+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 msgr2=0x7f39b4141b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.821+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39a0000f50 tx=0x7f39a0001a00 comp rx=0 tx=0).stop 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 shutdown_connections 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f399806af00 0x7f399806d3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b415a840 0x7f39b415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a4004850 0x7f39b4152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 --2- 172.21.15.114:0/3715464685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39b41416e0 0x7f39b4141b60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 >> 172.21.15.114:0/3715464685 conn(0x7f39b4071b90 msgr2=0x7f39b40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 shutdown_connections 2023-11-24T06:25:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.822+0000 7f39baf89700 1 -- 172.21.15.114:0/3715464685 wait complete. 2023-11-24T06:25:05.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:25:05.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:05.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.956+0000 7fb2bf6ab700 1 Processor -- start 2023-11-24T06:25:05.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.956+0000 7fb2bf6ab700 1 -- start start 2023-11-24T06:25:05.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bf6ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b815bdd0 0x7fb2b81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bf6ab700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb2b80d1c70 0x7fb2b80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bf6ab700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b80b22d0 con 0x7fb2b80b3370 2023-11-24T06:25:05.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bf6ab700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80b2f50 con 0x7fb2b80d1c70 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bf6ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80d2640 con 0x7fb2b815bdd0 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb2b80d1c70 0x7fb2b80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb2b80d1c70 0x7fb2b80d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39388/0 (socket says 172.21.15.114:39388) 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 -- 172.21.15.114:0/3791071377 learned_addr learned my addr 172.21.15.114:0/3791071377 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bd447700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b815bdd0 0x7fb2b81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 -- 172.21.15.114:0/3791071377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b815bdd0 msgr2=0x7fb2b81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b815bdd0 0x7fb2b81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 -- 172.21.15.114:0/3791071377 >> v1:172.21.15.172:6789/0 conn(0x7fb2b80b3370 legacy=0x7fb2b80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 -- 172.21.15.114:0/3791071377 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80b25b0 con 0x7fb2b80d1c70 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2bdc48700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b80d1c70 0x7fb2b80d2100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009e50 tx=0x7fb2b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16aefe1c1ad64a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2affff700 1 -- 172.21.15.114:0/3791071377 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000e070 con 0x7fb2b80d1c70 2023-11-24T06:25:05.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2affff700 1 -- 172.21.15.114:0/3791071377 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b000b9c0 con 0x7fb2b80d1c70 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.957+0000 7fb2affff700 1 -- 172.21.15.114:0/3791071377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000bd70 con 0x7fb2b80d1c70 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/3791071377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b80d1c70 msgr2=0x7fb2b80d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b80d1c70 0x7fb2b80d2100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009e50 tx=0x7fb2b000b330 comp rx=0 tx=0).stop 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/3791071377 shutdown_connections 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b80d1c70 0x7fb2b80d2100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/3791071377 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b815bdd0 0x7fb2b81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/3791071377 >> 172.21.15.114:0/3791071377 conn(0x7fb2b80731a0 msgr2=0x7fb2b809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/3791071377 shutdown_connections 2023-11-24T06:25:05.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.958+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/3791071377 wait complete. 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.959+0000 7fb2bf6ab700 1 Processor -- start 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.959+0000 7fb2bf6ab700 1 -- start start 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 0x7fb2b80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 0x7fb2b8157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8158580 con 0x7fb2b815bdd0 2023-11-24T06:25:05.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8144180 con 0x7fb2b80d1c70 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80b6910 con 0x7fb2b80b3370 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39394/0 (socket says 172.21.15.114:39394) 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 -- 172.21.15.114:0/510265106 learned_addr learned my addr 172.21.15.114:0/510265106 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bd447700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 0x7fb2b80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bcc46700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 0x7fb2b8157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 -- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 msgr2=0x7fb2b80c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 0x7fb2b80c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 -- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 msgr2=0x7fb2b8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 0x7fb2b8157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 -- 172.21.15.114:0/510265106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009b30 con 0x7fb2b815bdd0 2023-11-24T06:25:05.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bcc46700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 0x7fb2b8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bd447700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 0x7fb2b80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bdc48700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80716d0 tx=0x7fb2b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004960 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00099b0 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b001c680 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.960+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80b6bf0 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.961+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b80b71b0 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.961+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8067f60 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.962+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2b001c7e0 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.962+0000 7fb2adffb700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 0x7fb29c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.962+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb2b008b380 con 0x7fb2b815bdd0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.962+0000 7fb2bd447700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 0x7fb29c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:05.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.963+0000 7fb2bd447700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 0x7fb29c06d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b77f0 tx=0x7fb2a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:05.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:05.967+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2b0056440 con 0x7fb2b815bdd0 2023-11-24T06:25:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.203+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2b80d21e0 con 0x7fb2b815bdd0 2023-11-24T06:25:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.204+0000 7fb2adffb700 1 -- 172.21.15.114:0/510265106 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fb2b0059a60 con 0x7fb2b815bdd0 2023-11-24T06:25:06.206 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:06.206 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:25:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 msgr2=0x7fb29c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 0x7fb29c06d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b77f0 tx=0x7fb2a8008040 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 msgr2=0x7fb2b8158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80716d0 tx=0x7fb2b0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 shutdown_connections 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb29c06aed0 0x7fb29c06d390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2b815bdd0 0x7fb2b8158040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80b3370 0x7fb2b80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 --2- 172.21.15.114:0/510265106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b80d1c70 0x7fb2b8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 >> 172.21.15.114:0/510265106 conn(0x7fb2b80731a0 msgr2=0x7fb2b80b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 shutdown_connections 2023-11-24T06:25:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.206+0000 7fb2bf6ab700 1 -- 172.21.15.114:0/510265106 wait complete. 2023-11-24T06:25:06.210 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:25:06.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:06.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.339+0000 7f0bf2949700 1 Processor -- start 2023-11-24T06:25:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.339+0000 7f0bf2949700 1 -- start start 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bec158670 0x7f0bec0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0bec158a50 0x7f0bec0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec162a30 0x7f0bec0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0cfff0 con 0x7f0bec158a50 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0cfd10 con 0x7f0bec158670 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf2949700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec161f50 con 0x7f0bec162a30 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0bec158a50 0x7f0bec0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0bec158a50 0x7f0bec0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39400/0 (socket says 172.21.15.114:39400) 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 -- 172.21.15.114:0/2876537702 learned_addr learned my addr 172.21.15.114:0/2876537702 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bebfff700 1 --2- 172.21.15.114:0/2876537702 >> v2:172.21.15.172:3300/0 conn(0x7f0bec158670 0x7f0bec0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0bf0ee6700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec162a30 0x7f0bec0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 -- 172.21.15.114:0/2876537702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec162a30 msgr2=0x7f0bec0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec162a30 0x7f0bec0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 -- 172.21.15.114:0/2876537702 >> v2:172.21.15.172:3300/0 conn(0x7f0bec158670 msgr2=0x7f0bec0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/2876537702 >> v2:172.21.15.172:3300/0 conn(0x7f0bec158670 0x7f0bec0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 -- 172.21.15.114:0/2876537702 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec162230 con 0x7f0bec158a50 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158a50 0x7f0bec0c98b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0bec09e5c0 tx=0x7f0bdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a89b020c0ef406b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.340+0000 7f0beaffd700 1 -- 172.21.15.114:0/2876537702 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00b870 con 0x7f0bec158a50 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0beaffd700 1 -- 172.21.15.114:0/2876537702 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc00b9d0 con 0x7f0bec158a50 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0beaffd700 1 -- 172.21.15.114:0/2876537702 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00bd60 con 0x7f0bec158a50 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 -- 172.21.15.114:0/2876537702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158a50 msgr2=0x7f0bec0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158a50 0x7f0bec0c98b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0bec09e5c0 tx=0x7f0bdc009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 -- 172.21.15.114:0/2876537702 shutdown_connections 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec162a30 0x7f0bec0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 --2- 172.21.15.114:0/2876537702 >> v2:172.21.15.172:3300/0 conn(0x7f0bec158670 0x7f0bec0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 --2- 172.21.15.114:0/2876537702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158a50 0x7f0bec0c98b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 -- 172.21.15.114:0/2876537702 >> 172.21.15.114:0/2876537702 conn(0x7f0bec0b57b0 msgr2=0x7f0bec0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 -- 172.21.15.114:0/2876537702 shutdown_connections 2023-11-24T06:25:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.341+0000 7f0bf2949700 1 -- 172.21.15.114:0/2876537702 wait complete. 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.342+0000 7f0bf2949700 1 Processor -- start 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.342+0000 7f0bf2949700 1 -- start start 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec158a50 0x7f0bec0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 0x7f0bec0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec160390 con 0x7f0bec158670 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec160670 con 0x7f0bec162a30 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf2949700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec160950 con 0x7f0bec158a50 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bebfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39404/0 (socket says 172.21.15.114:39404) 2023-11-24T06:25:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bebfff700 1 -- 172.21.15.114:0/1700357459 learned_addr learned my addr 172.21.15.114:0/1700357459 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec158a50 0x7f0bec0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.343+0000 7f0bf0ee6700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 0x7f0bec0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 -- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec158a50 msgr2=0x7f0bec0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec158a50 0x7f0bec0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 -- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 msgr2=0x7f0bec0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 0x7f0bec0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 -- 172.21.15.114:0/1700357459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009550 con 0x7f0bec158670 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bf0ee6700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 0x7f0bec0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bebfff700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e810 tx=0x7f0be000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0013610 con 0x7f0bec158670 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0013790 con 0x7f0bec158670 2023-11-24T06:25:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec160c30 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0013b40 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.344+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec151610 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.345+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec0ad3c0 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.345+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0be0013ca0 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.346+0000 7f0be8ff9700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 0x7f0bd806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.346+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0be00891d0 con 0x7f0bec158670 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.346+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 0x7f0bd806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.346+0000 7f0beb7fe700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 0x7f0bd806d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bec09e5c0 tx=0x7f0bdc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.350+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0be0054200 con 0x7f0bec158670 2023-11-24T06:25:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.582+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bec0b0a10 con 0x7f0bec158670 2023-11-24T06:25:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.583+0000 7f0be8ff9700 1 -- 172.21.15.114:0/1700357459 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f0be0009440 con 0x7f0bec158670 2023-11-24T06:25:06.585 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:06.586 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/24005028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":24005028},{"type":"v1","addr":"172.21.15.114:6836","nonce":24005028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/784634998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":784634998},{"type":"v1","addr":"172.21.15.172:6833","nonce":784634998}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/989720355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":989720355},{"type":"v1","addr":"172.21.15.172:6835","nonce":989720355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:24:19.243682+0000","modified":"2023-11-24T06:24:22.264928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/2364623566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2364623566},{"type":"v1","addr":"172.21.15.114:6837","nonce":2364623566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 msgr2=0x7f0bd806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 0x7f0bd806d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bec09e5c0 tx=0x7f0bdc008030 comp rx=0 tx=0).stop 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 msgr2=0x7f0bec1510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e810 tx=0x7f0be000eb20 comp rx=0 tx=0).stop 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 shutdown_connections 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0bd806aed0 0x7f0bd806d390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bec162a30 0x7f0bec0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bec158670 0x7f0bec1510d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 --2- 172.21.15.114:0/1700357459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bec158a50 0x7f0bec0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 >> 172.21.15.114:0/1700357459 conn(0x7f0bec0b57b0 msgr2=0x7f0bec0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 shutdown_connections 2023-11-24T06:25:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.585+0000 7f0bf2949700 1 -- 172.21.15.114:0/1700357459 wait complete. 2023-11-24T06:25:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2023-11-24T06:25:06.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.718+0000 7f26848f5700 1 Processor -- start 2023-11-24T06:25:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.718+0000 7f26848f5700 1 -- start start 2023-11-24T06:25:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f26848f5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f268015e080 0x7f26800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f26848f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268006c710 0x7f26800afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f26848f5700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26800ae430 con 0x7f26800af4d0 2023-11-24T06:25:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f26848f5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26800af0b0 con 0x7f268015e080 2023-11-24T06:25:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f26848f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26800b05a0 con 0x7f268006c710 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267dd9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f26800af4d0 0x7f268006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55120/0 (socket says 172.21.15.114:55120) 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267dd9b700 1 -- 172.21.15.114:0/1865683709 learned_addr learned my addr 172.21.15.114:0/1865683709 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69268089 0 0) 0x7f26800ae430 con 0x7f26800af4d0 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26680035b0 con 0x7f26800af4d0 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268006c710 0x7f26800afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.719+0000 7f267e59c700 1 --2- 172.21.15.114:0/1865683709 >> v2:172.21.15.172:3300/0 conn(0x7f268015e080 0x7f26800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295103061 0 0) 0x7f26680035b0 con 0x7f26800af4d0 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268006c710 msgr2=0x7f26800afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268006c710 0x7f26800afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 >> v2:172.21.15.172:3300/0 conn(0x7f268015e080 msgr2=0x7f26800b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 --2- 172.21.15.114:0/1865683709 >> v2:172.21.15.172:3300/0 conn(0x7f268015e080 0x7f26800b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800ae710 con 0x7f26800af4d0 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f2670001670 con 0x7f26800af4d0 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267e59c700 1 --2- 172.21.15.114:0/1865683709 >> v2:172.21.15.172:3300/0 conn(0x7f268015e080 0x7f26800b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 >> v1:172.21.15.114:6789/0 conn(0x7f26800af4d0 legacy=0x7f268006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2668003a30 0x7f2668003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2668004850 0x7f2668004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f266800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800af0b0 con 0x7f2668005210 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800b05a0 con 0x7f2668003a30 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266800c080 con 0x7f2668004850 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268006c710 0x7f26800afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f266800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267e59c700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2668003a30 0x7f2668003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267dd9b700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2668004850 0x7f2668004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 -- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2668004850 msgr2=0x7f2668004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2668004850 0x7f2668004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 -- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2668003a30 msgr2=0x7f2668003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2668003a30 0x7f2668003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 -- 172.21.15.114:0/1865683709 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26680035b0 con 0x7f2668005210 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267ed9d700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f266800ba10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f268006dbd0 tx=0x7f2668005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73af396e9b0be459 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678007dc0 con 0x7f2668005210 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800f040 con 0x7f2668005210 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.720+0000 7f267d59a700 1 -- 172.21.15.114:0/1865683709 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678003c60 con 0x7f2668005210 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 -- 172.21.15.114:0/1865683709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 msgr2=0x7f266800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 --2- 172.21.15.114:0/1865683709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f266800ba10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f268006dbd0 tx=0x7f2668005880 comp rx=0 tx=0).stop 2023-11-24T06:25:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f267ed9d700 1 -- 172.21.15.114:0/1865683709 reap_dead start 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 -- 172.21.15.114:0/1865683709 shutdown_connections 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 -- 172.21.15.114:0/1865683709 >> 172.21.15.114:0/1865683709 conn(0x7f2680071b90 msgr2=0x7f2680071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 -- 172.21.15.114:0/1865683709 shutdown_connections 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.721+0000 7f26848f5700 1 -- 172.21.15.114:0/1865683709 wait complete. 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.722+0000 7f26848f5700 1 Processor -- start 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.722+0000 7f26848f5700 1 -- start start 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680156650 0x7f26800cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26800b4090 0x7f26800b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800b4a60 con 0x7f2668005210 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800cf400 con 0x7f2680156650 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f26848f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26800ce7f0 con 0x7f26800b4090 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f267e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f267e59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39424/0 (socket says 172.21.15.114:39424) 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.723+0000 7f267e59c700 1 -- 172.21.15.114:0/3262752832 learned_addr learned my addr 172.21.15.114:0/3262752832 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267dd9b700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680156650 0x7f26800cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 -- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26800b4090 msgr2=0x7f26800b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26800b4090 0x7f26800b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 -- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680156650 msgr2=0x7f26800cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680156650 0x7f26800cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 -- 172.21.15.114:0/3262752832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26800cead0 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f267e59c700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2674011900 tx=0x7f2674011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674004230 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2674004390 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26800cedb0 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26740036a0 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.724+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2680156ad0 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.725+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26800753f0 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.725+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2674003800 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.725+0000 7f266effd700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 0x7f266006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.726+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2674089850 con 0x7f2668005210 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.726+0000 7f267dd9b700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 0x7f266006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.726+0000 7f267dd9b700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 0x7f266006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26800af950 tx=0x7f2670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:06.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.730+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2674054880 con 0x7f2668005210 2023-11-24T06:25:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.924+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26800cf540 con 0x7f2668005210 2023-11-24T06:25:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.926+0000 7f266effd700 1 -- 172.21.15.114:0/3262752832 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f267400f040 con 0x7f2668005210 2023-11-24T06:25:06.928 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:06.928 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:24:59.029214+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:24:17.780078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:24:18.203282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 msgr2=0x7f266006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 0x7f266006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26800af950 tx=0x7f2670009040 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 msgr2=0x7f26801f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2674011900 tx=0x7f2674011cc0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 shutdown_connections 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f266006aed0 0x7f266006d390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26800b4090 0x7f26800b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2668005210 0x7f26801f3860 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 --2- 172.21.15.114:0/3262752832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680156650 0x7f26800cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 >> 172.21.15.114:0/3262752832 conn(0x7f2680071b90 msgr2=0x7f26800b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 shutdown_connections 2023-11-24T06:25:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:06.928+0000 7f26848f5700 1 -- 172.21.15.114:0/3262752832 wait complete. 2023-11-24T06:25:06.943 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:25:07.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.062+0000 7fdcbb9f1700 1 Processor -- start 2023-11-24T06:25:07.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.062+0000 7fdcbb9f1700 1 -- start start 2023-11-24T06:25:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb415bdd0 0x7fdcb41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdcb40b3370 0x7fdcb40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdcb40d1c70 0x7fdcb40d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb40b2f50 con 0x7fdcb40d1c70 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb40d2660 con 0x7fdcb40b3370 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcbb9f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb40b2030 con 0x7fdcb415bdd0 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdcb40d1c70 0x7fdcb40d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdcb40d1c70 0x7fdcb40d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39432/0 (socket says 172.21.15.114:39432) 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/2182810273 learned_addr learned my addr 172.21.15.114:0/2182810273 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb978d700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb415bdd0 0x7fdcb41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb8f8c700 1 --2- 172.21.15.114:0/2182810273 >> v2:172.21.15.172:3300/0 conn(0x7fdcb40b3370 0x7fdcb40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/2182810273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb415bdd0 msgr2=0x7fdcb41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb415bdd0 0x7fdcb41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/2182810273 >> v2:172.21.15.172:3300/0 conn(0x7fdcb40b3370 msgr2=0x7fdcb40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/2182810273 >> v2:172.21.15.172:3300/0 conn(0x7fdcb40b3370 0x7fdcb40b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/2182810273 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40b22d0 con 0x7fdcb40d1c70 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.063+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb40d1c70 0x7fdcb40d2120 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40cf7a0 tx=0x7fdcac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36caceb89e68e655 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcabfff700 1 -- 172.21.15.114:0/2182810273 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00e070 con 0x7fdcb40d1c70 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcabfff700 1 -- 172.21.15.114:0/2182810273 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac00b9c0 con 0x7fdcb40d1c70 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcabfff700 1 -- 172.21.15.114:0/2182810273 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00bd70 con 0x7fdcb40d1c70 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/2182810273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb40d1c70 msgr2=0x7fdcb40d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb40d1c70 0x7fdcb40d2120 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40cf7a0 tx=0x7fdcac00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/2182810273 shutdown_connections 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb415bdd0 0x7fdcb41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/2182810273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb40d1c70 0x7fdcb40d2120 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/2182810273 >> v2:172.21.15.172:3300/0 conn(0x7fdcb40b3370 0x7fdcb40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/2182810273 >> 172.21.15.114:0/2182810273 conn(0x7fdcb40731a0 msgr2=0x7fdcb409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/2182810273 shutdown_connections 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.064+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/2182810273 wait complete. 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.065+0000 7fdcbb9f1700 1 Processor -- start 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 -- start start 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb40b3370 0x7fdcb4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb40d1c70 0x7fdcb40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41e5c00 con 0x7fdcb415bdd0 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41e5ee0 con 0x7fdcb40b3370 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcbb9f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb41e61c0 con 0x7fdcb40d1c70 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcb9f8e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcb9f8e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39442/0 (socket says 172.21.15.114:39442) 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/235327266 learned_addr learned my addr 172.21.15.114:0/235327266 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcb978d700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb40b3370 0x7fdcb4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.066+0000 7fdcb8f8c700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb40d1c70 0x7fdcb40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb40d1c70 msgr2=0x7fdcb40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb40d1c70 0x7fdcb40cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb40b3370 msgr2=0x7fdcb4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb40b3370 0x7fdcb4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 -- 172.21.15.114:0/235327266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac009b30 con 0x7fdcb415bdd0 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcb9f8e700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40716d0 tx=0x7fdcac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0099b0 con 0x7fdcb415bdd0 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac004250 con 0x7fdcb415bdd0 2023-11-24T06:25:07.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb41e64a0 con 0x7fdcb415bdd0 2023-11-24T06:25:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac004600 con 0x7fdcb415bdd0 2023-11-24T06:25:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.067+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb41e5790 con 0x7fdcb415bdd0 2023-11-24T06:25:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.068+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb4067f60 con 0x7fdcb415bdd0 2023-11-24T06:25:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.069+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcac004760 con 0x7fdcb415bdd0 2023-11-24T06:25:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.069+0000 7fdca9ffb700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 0x7fdc9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.069+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdcac08aac0 con 0x7fdcb415bdd0 2023-11-24T06:25:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.069+0000 7fdcb978d700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 0x7fdc9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.069+0000 7fdcb978d700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 0x7fdc9806d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdca4006b20 tx=0x7fdca4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.074+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcac055c10 con 0x7fdcb415bdd0 2023-11-24T06:25:07.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.308+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdcb41e5d40 con 0x7fdcb415bdd0 2023-11-24T06:25:07.311 INFO:tasks.ceph.mon.a.smithi114.stderr:2023-11-24T06:25:07.308+0000 7f8fd8393700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:25:07.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.310+0000 7fdca9ffb700 1 -- 172.21.15.114:0/235327266 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdcb41e5d40 con 0x7fdcb415bdd0 2023-11-24T06:25:07.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.312+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 msgr2=0x7fdc9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.312+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 0x7fdc9806d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdca4006b20 tx=0x7fdca4008040 comp rx=0 tx=0).stop 2023-11-24T06:25:07.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.312+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 msgr2=0x7fdcb41e6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.312+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40716d0 tx=0x7fdcac004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 shutdown_connections 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc9806aed0 0x7fdc9806d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb415bdd0 0x7fdcb41e6f50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb40b3370 0x7fdcb4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 --2- 172.21.15.114:0/235327266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdcb40d1c70 0x7fdcb40cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 >> 172.21.15.114:0/235327266 conn(0x7fdcb40731a0 msgr2=0x7fdcb406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 shutdown_connections 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.313+0000 7fdcbb9f1700 1 -- 172.21.15.114:0/235327266 wait complete. 2023-11-24T06:25:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:25:07.328 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:25:07.353 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:25:07.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.449+0000 7f4f5e674700 1 Processor -- start 2023-11-24T06:25:07.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.449+0000 7f4f5e674700 1 -- start start 2023-11-24T06:25:07.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5e674700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f580c6c30 0x7f4f580cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5e674700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f580cef40 0x7f4f580cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5e674700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580d0140 con 0x7f4f5815bf50 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5e674700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5815c330 con 0x7f4f580cef40 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5e674700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580cf3d0 con 0x7f4f580c6c30 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5cc11700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f580cef40 0x7f4f580cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5cc11700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f580cef40 0x7f4f580cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39458/0 (socket says 172.21.15.114:39458) 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f5cc11700 1 -- 172.21.15.114:0/815347244 learned_addr learned my addr 172.21.15.114:0/815347244 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.450+0000 7f4f4ffff700 1 --2- 172.21.15.114:0/815347244 >> v2:172.21.15.172:3300/0 conn(0x7f4f580c6c30 0x7f4f580cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f5cc11700 1 -- 172.21.15.114:0/815347244 >> v2:172.21.15.172:3300/0 conn(0x7f4f580c6c30 msgr2=0x7f4f580cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f5cc11700 1 --2- 172.21.15.114:0/815347244 >> v2:172.21.15.172:3300/0 conn(0x7f4f580c6c30 0x7f4f580cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f5cc11700 1 -- 172.21.15.114:0/815347244 >> v1:172.21.15.172:6789/0 conn(0x7f4f5815bf50 legacy=0x7f4f580ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f5cc11700 1 -- 172.21.15.114:0/815347244 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580d03e0 con 0x7f4f580cef40 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f5cc11700 1 --2- 172.21.15.114:0/815347244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f580cc6c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f58072420 tx=0x7f4f5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab75f4b6ec551ef7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f577fe700 1 -- 172.21.15.114:0/815347244 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000e070 con 0x7f4f580cef40 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f577fe700 1 -- 172.21.15.114:0/815347244 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5000b910 con 0x7f4f580cef40 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.451+0000 7f4f577fe700 1 -- 172.21.15.114:0/815347244 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000bcc0 con 0x7f4f580cef40 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 -- 172.21.15.114:0/815347244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 msgr2=0x7f4f580cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 --2- 172.21.15.114:0/815347244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f580cc6c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f58072420 tx=0x7f4f5000b330 comp rx=0 tx=0).stop 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 -- 172.21.15.114:0/815347244 shutdown_connections 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 --2- 172.21.15.114:0/815347244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f580cc6c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 --2- 172.21.15.114:0/815347244 >> v2:172.21.15.172:3300/0 conn(0x7f4f580c6c30 0x7f4f580cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 -- 172.21.15.114:0/815347244 >> 172.21.15.114:0/815347244 conn(0x7f4f580b3780 msgr2=0x7f4f580b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 -- 172.21.15.114:0/815347244 shutdown_connections 2023-11-24T06:25:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.452+0000 7f4f5e674700 1 -- 172.21.15.114:0/815347244 wait complete. 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.453+0000 7f4f5e674700 1 Processor -- start 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.453+0000 7f4f5e674700 1 -- start start 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 0x7f4f58154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 0x7f4f580d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5809d0d0 con 0x7f4f580cef40 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58155170 con 0x7f4f580c6c30 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5e674700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5809c4c0 con 0x7f4f5815bf50 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f4ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f4ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39466/0 (socket says 172.21.15.114:39466) 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f4ffff700 1 -- 172.21.15.114:0/3060116662 learned_addr learned my addr 172.21.15.114:0/3060116662 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f5cc11700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 0x7f4f580d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 0x7f4f58154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f4ffff700 1 -- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 msgr2=0x7f4f580d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.454+0000 7f4f4ffff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 0x7f4f580d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f4ffff700 1 -- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 msgr2=0x7f4f58154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f4ffff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 0x7f4f58154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f4ffff700 1 -- 172.21.15.114:0/3060116662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f50009b30 con 0x7f4f580cef40 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f5cc11700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 0x7f4f580d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 0x7f4f58154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f4ffff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000e9b0 tx=0x7f4f4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40009710 con 0x7f4f580cef40 2023-11-24T06:25:07.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4000c850 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5809c7a0 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000cbc0 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.455+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5809cd60 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.456+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5809d3b0 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.456+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f4000cd20 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.457+0000 7f4f557fa700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 0x7f4f3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.457+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f4f40089750 con 0x7f4f580cef40 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.457+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 0x7f4f3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:07.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.457+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 0x7f4f3806d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000f60 tx=0x7f4f48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:07.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.461+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f40054730 con 0x7f4f580cef40 2023-11-24T06:25:07.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:07.695+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f5809d210 con 0x7f4f580cef40 2023-11-24T06:25:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.319+0000 7f4f557fa700 1 -- 172.21.15.114:0/3060116662 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4f5809d210 con 0x7f4f580cef40 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 msgr2=0x7f4f3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 0x7f4f3806d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000f60 tx=0x7f4f48009040 comp rx=0 tx=0).stop 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 msgr2=0x7f4f5809bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000e9b0 tx=0x7f4f4000ed70 comp rx=0 tx=0).stop 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 shutdown_connections 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f3806af00 0x7f4f3806d3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f5815bf50 0x7f4f580d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f580c6c30 0x7f4f58154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 --2- 172.21.15.114:0/3060116662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f580cef40 0x7f4f5809bb70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 >> 172.21.15.114:0/3060116662 conn(0x7f4f580b3780 msgr2=0x7f4f5806efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:08.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 shutdown_connections 2023-11-24T06:25:08.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.323+0000 7f4f5e674700 1 -- 172.21.15.114:0/3060116662 wait complete. 2023-11-24T06:25:08.340 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:25:08.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.505+0000 7efd0087e700 1 Processor -- start 2023-11-24T06:25:08.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.505+0000 7efd0087e700 1 -- start start 2023-11-24T06:25:08.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efd0087e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efcfc15e020 0x7efcfc0b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efd0087e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efcfc0cfae0 0x7efcfc0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efd0087e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0b1b80 con 0x7efcfc0b1720 2023-11-24T06:25:08.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efd0087e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc161c80 con 0x7efcfc15e020 2023-11-24T06:25:08.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efd0087e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc161f40 con 0x7efcfc0cfae0 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efcfa59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efcfc15e020 0x7efcfc0b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efcfa59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efcfc15e020 0x7efcfc0b11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39476/0 (socket says 172.21.15.114:39476) 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.506+0000 7efcfa59c700 1 -- 172.21.15.114:0/392508558 learned_addr learned my addr 172.21.15.114:0/392508558 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfa59c700 1 -- 172.21.15.114:0/392508558 >> v2:172.21.15.172:3300/0 conn(0x7efcfc0cfae0 msgr2=0x7efcfc0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfad9d700 1 --2- 172.21.15.114:0/392508558 >> v2:172.21.15.172:3300/0 conn(0x7efcfc0cfae0 0x7efcfc0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfa59c700 1 --2- 172.21.15.114:0/392508558 >> v2:172.21.15.172:3300/0 conn(0x7efcfc0cfae0 0x7efcfc0af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfa59c700 1 -- 172.21.15.114:0/392508558 >> v1:172.21.15.172:6789/0 conn(0x7efcfc0b1720 legacy=0x7efcfc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfa59c700 1 -- 172.21.15.114:0/392508558 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc162220 con 0x7efcfc15e020 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcfa59c700 1 --2- 172.21.15.114:0/392508558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc15e020 0x7efcfc0b11e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0cdd00 tx=0x7efcf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66b14dd06a4edc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcf959a700 1 -- 172.21.15.114:0/392508558 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004230 con 0x7efcfc15e020 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcf959a700 1 -- 172.21.15.114:0/392508558 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf0004390 con 0x7efcfc15e020 2023-11-24T06:25:08.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.507+0000 7efcf959a700 1 -- 172.21.15.114:0/392508558 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00036a0 con 0x7efcfc15e020 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 -- 172.21.15.114:0/392508558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc15e020 msgr2=0x7efcfc0b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 --2- 172.21.15.114:0/392508558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc15e020 0x7efcfc0b11e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0cdd00 tx=0x7efcf0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 -- 172.21.15.114:0/392508558 shutdown_connections 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 --2- 172.21.15.114:0/392508558 >> v2:172.21.15.172:3300/0 conn(0x7efcfc0cfae0 0x7efcfc0af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 --2- 172.21.15.114:0/392508558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc15e020 0x7efcfc0b11e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 -- 172.21.15.114:0/392508558 >> 172.21.15.114:0/392508558 conn(0x7efcfc072710 msgr2=0x7efcfc072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 -- 172.21.15.114:0/392508558 shutdown_connections 2023-11-24T06:25:08.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.508+0000 7efd0087e700 1 -- 172.21.15.114:0/392508558 wait complete. 2023-11-24T06:25:08.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.509+0000 7efd0087e700 1 Processor -- start 2023-11-24T06:25:08.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.509+0000 7efd0087e700 1 -- start start 2023-11-24T06:25:08.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 0x7efcfc0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 0x7efcfc0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0b0320 con 0x7efcfc0cfae0 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0adbf0 con 0x7efcfc15e020 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efd0087e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0ade60 con 0x7efcfc0b1720 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39482/0 (socket says 172.21.15.114:39482) 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 -- 172.21.15.114:0/1360931254 learned_addr learned my addr 172.21.15.114:0/1360931254 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcfa59c700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 0x7efcfc0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcfad9d700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 0x7efcfc0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 -- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 msgr2=0x7efcfc0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 0x7efcfc0c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 -- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 msgr2=0x7efcfc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 0x7efcfc0b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 -- 172.21.15.114:0/1360931254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf0009550 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcfa59c700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 0x7efcfc0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcfad9d700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 0x7efcfc0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.510+0000 7efcf9d9b700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efcec00b4a0 tx=0x7efcec00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.511+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec011830 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.511+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec0119b0 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.511+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0ae140 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.511+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec011d60 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.511+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc0ad170 con 0x7efcfc0cfae0 2023-11-24T06:25:08.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.512+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc067f60 con 0x7efcfc0cfae0 2023-11-24T06:25:08.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.512+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcec01a400 con 0x7efcfc0cfae0 2023-11-24T06:25:08.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.512+0000 7efceaffd700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 0x7efcdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:08.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.513+0000 7efcfa59c700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 0x7efcdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:08.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.513+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7efcec089670 con 0x7efcfc0cfae0 2023-11-24T06:25:08.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.513+0000 7efcfa59c700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 0x7efcdc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009130 tx=0x7efcf0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:08.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.517+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcec0546e0 con 0x7efcfc0cfae0 2023-11-24T06:25:08.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:08.708+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efcfc0cd590 con 0x7efcfc0cfae0 2023-11-24T06:25:09.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.320+0000 7efceaffd700 1 -- 172.21.15.114:0/1360931254 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7efcec057d00 con 0x7efcfc0cfae0 2023-11-24T06:25:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 msgr2=0x7efcdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 0x7efcdc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009130 tx=0x7efcf0008030 comp rx=0 tx=0).stop 2023-11-24T06:25:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 msgr2=0x7efcfc15a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efcec00b4a0 tx=0x7efcec00b860 comp rx=0 tx=0).stop 2023-11-24T06:25:09.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 shutdown_connections 2023-11-24T06:25:09.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efcdc06aed0 0x7efcdc06d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc15e020 0x7efcfc0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efcfc0b1720 0x7efcfc0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 --2- 172.21.15.114:0/1360931254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcfc0cfae0 0x7efcfc15a310 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.323+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 >> 172.21.15.114:0/1360931254 conn(0x7efcfc072710 msgr2=0x7efcfc160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:09.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.324+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 shutdown_connections 2023-11-24T06:25:09.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.324+0000 7efd0087e700 1 -- 172.21.15.114:0/1360931254 wait complete. 2023-11-24T06:25:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:25:09.341 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:25:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.468+0000 7fb9eb22e700 1 Processor -- start 2023-11-24T06:25:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.468+0000 7fb9eb22e700 1 -- start start 2023-11-24T06:25:09.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9eb22e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9e4144600 0x7fb9e40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9eb22e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb9e406f480 0x7fb9e406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9eb22e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e406b200 con 0x7fb9e406e830 2023-11-24T06:25:09.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9eb22e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e406aff0 con 0x7fb9e406f480 2023-11-24T06:25:09.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9eb22e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e406b4e0 con 0x7fb9e4144600 2023-11-24T06:25:09.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb9e406f480 0x7fb9e406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:09.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e97cb700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb9e406e830 0x7fb9e40cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55200/0 (socket says 172.21.15.114:55200) 2023-11-24T06:25:09.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e97cb700 1 -- 172.21.15.114:0/2276168970 learned_addr learned my addr 172.21.15.114:0/2276168970 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:09.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb9e406f480 0x7fb9e406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39490/0 (socket says 172.21.15.114:39490) 2023-11-24T06:25:09.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 -- 172.21.15.114:0/2276168970 >> v2:172.21.15.172:3300/0 conn(0x7fb9e4144600 msgr2=0x7fb9e40b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:09.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 --2- 172.21.15.114:0/2276168970 >> v2:172.21.15.172:3300/0 conn(0x7fb9e4144600 0x7fb9e40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 -- 172.21.15.114:0/2276168970 >> v1:172.21.15.114:6789/0 conn(0x7fb9e406e830 legacy=0x7fb9e40cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 -- 172.21.15.114:0/2276168970 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e406b7c0 con 0x7fb9e406f480 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e37fe700 1 -- 172.21.15.114:0/2276168970 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746266073 0 0) 0x7fb9e406b200 con 0x7fb9e406e830 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e3fff700 1 --2- 172.21.15.114:0/2276168970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e406c2a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40734a0 tx=0x7fb9d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4090c030bc31a858 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.469+0000 7fb9e37fe700 1 -- 172.21.15.114:0/2276168970 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0011830 con 0x7fb9e406f480 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9e37fe700 1 -- 172.21.15.114:0/2276168970 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0011990 con 0x7fb9e406f480 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9e37fe700 1 -- 172.21.15.114:0/2276168970 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0011d20 con 0x7fb9e406f480 2023-11-24T06:25:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 -- 172.21.15.114:0/2276168970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 msgr2=0x7fb9e406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/2276168970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e406c2a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40734a0 tx=0x7fb9d0009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 -- 172.21.15.114:0/2276168970 shutdown_connections 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/2276168970 >> v2:172.21.15.172:3300/0 conn(0x7fb9e4144600 0x7fb9e40b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/2276168970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e406c2a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 -- 172.21.15.114:0/2276168970 >> 172.21.15.114:0/2276168970 conn(0x7fb9e40d0b40 msgr2=0x7fb9e40d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:09.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 -- 172.21.15.114:0/2276168970 shutdown_connections 2023-11-24T06:25:09.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.470+0000 7fb9eb22e700 1 -- 172.21.15.114:0/2276168970 wait complete. 2023-11-24T06:25:09.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.471+0000 7fb9eb22e700 1 Processor -- start 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.471+0000 7fb9eb22e700 1 -- start start 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9e406e830 0x7fb9e41e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 0x7fb9e4141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e415f030 con 0x7fb9e406f480 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41f55a0 con 0x7fb9e4144600 2023-11-24T06:25:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9eb22e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41f5880 con 0x7fb9e406e830 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9e3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9e3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39496/0 (socket says 172.21.15.114:39496) 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9e3fff700 1 -- 172.21.15.114:0/3035586225 learned_addr learned my addr 172.21.15.114:0/3035586225 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.472+0000 7fb9e8fca700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9e406e830 0x7fb9e41e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 -- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9e406e830 msgr2=0x7fb9e41e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9e406e830 0x7fb9e41e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 -- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 msgr2=0x7fb9e4141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e97cb700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 0x7fb9e4141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 0x7fb9e4141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 -- 172.21.15.114:0/3035586225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00094a0 con 0x7fb9e406f480 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e97cb700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 0x7fb9e4141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e3fff700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00090b0 tx=0x7fb9d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00036a0 con 0x7fb9e406f480 2023-11-24T06:25:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d001f070 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e41f5b60 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003a40 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.473+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e41409d0 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.474+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e4067f60 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.474+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9d0003ba0 con 0x7fb9e406f480 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.475+0000 7fb9e17fa700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 0x7fb9c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.475+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fb9d0021070 con 0x7fb9e406f480 2023-11-24T06:25:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.475+0000 7fb9e8fca700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 0x7fb9c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.475+0000 7fb9e8fca700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 0x7fb9c806d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800a810 tx=0x7fb9d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.479+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9d00561c0 con 0x7fb9e406f480 2023-11-24T06:25:09.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:09.671+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb9e41f6080 con 0x7fb9e406f480 2023-11-24T06:25:10.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.327+0000 7fb9e17fa700 1 -- 172.21.15.114:0/3035586225 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb9e41f6080 con 0x7fb9e406f480 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 msgr2=0x7fb9c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 0x7fb9c806d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800a810 tx=0x7fb9d8008040 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 msgr2=0x7fb9e415b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00090b0 tx=0x7fb9d0004750 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 shutdown_connections 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb9c806aed0 0x7fb9c806d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9e4144600 0x7fb9e4141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb9e406e830 0x7fb9e41e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 --2- 172.21.15.114:0/3035586225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9e406f480 0x7fb9e415b240 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 >> 172.21.15.114:0/3035586225 conn(0x7fb9e40d0b40 msgr2=0x7fb9e40b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:10.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 shutdown_connections 2023-11-24T06:25:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.330+0000 7fb9eb22e700 1 -- 172.21.15.114:0/3035586225 wait complete. 2023-11-24T06:25:10.335 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2023-11-24T06:25:10.348 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:25:10.348 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:25:10.348 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:25:10.348 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:10.349 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:25:10.349 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:25:10.349 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:25:10.349 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:10.349 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:25:10.349 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:25:10.349 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:25:10.349 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:10.350 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:25:10.350 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:25:10.350 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:25:10.350 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:10.359 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:25:10.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:25:10.360 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:25:10.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:25:10.365 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:25:10.365 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:25:10.366 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:25:10.366 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:25:10.401 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:25:10.402 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:10.402 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:25:10.402 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:10.407 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:25:10.408 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:10.408 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:25:10.408 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:10.408 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:25:10.481 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2023-11-24T06:25:10.481 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:25:10.492 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-11-24T06:25:10.493 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-11-24T06:25:10.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.533+0000 7fcd97d8f700 1 Processor -- start 2023-11-24T06:25:10.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.534+0000 7fcd97d8f700 1 -- start start 2023-11-24T06:25:10.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.534+0000 7fcd97d8f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd90158a50 0x7fcd900c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:10.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.534+0000 7fcd97d8f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd900cfff0 con 0x7fcd90162a30 2023-11-24T06:25:10.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.534+0000 7fcd97d8f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd900cfd10 con 0x7fcd90158670 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.534+0000 7fcd97d8f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900629b0 con 0x7fcd90158a50 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9532a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd90158a50 0x7fcd900c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9632c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcd90162a30 0x7fcd900ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55238/0 (socket says 172.21.15.114:55238) 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9632c700 1 -- 172.21.15.114:0/902736346 learned_addr learned my addr 172.21.15.114:0/902736346 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9532a700 1 -- 172.21.15.114:0/902736346 >> v1:172.21.15.172:6789/0 conn(0x7fcd90158670 legacy=0x7fcd900622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9532a700 1 -- 172.21.15.114:0/902736346 >> v1:172.21.15.114:6789/0 conn(0x7fcd90162a30 legacy=0x7fcd900ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd94b29700 1 -- 172.21.15.114:0/902736346 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173394929 0 0) 0x7fcd900cfff0 con 0x7fcd90162a30 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9532a700 1 -- 172.21.15.114:0/902736346 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90161f50 con 0x7fcd90158a50 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd9532a700 1 --2- 172.21.15.114:0/902736346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90158a50 0x7fcd900c98b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009840 tx=0x7fcd8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729b652525aee97f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd94b29700 1 -- 172.21.15.114:0/902736346 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c011830 con 0x7fcd90158a50 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd94b29700 1 -- 172.21.15.114:0/902736346 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c011990 con 0x7fcd90158a50 2023-11-24T06:25:10.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.535+0000 7fcd94b29700 1 -- 172.21.15.114:0/902736346 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c011d20 con 0x7fcd90158a50 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 -- 172.21.15.114:0/902736346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90158a50 msgr2=0x7fcd900c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/902736346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90158a50 0x7fcd900c98b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009840 tx=0x7fcd8c009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 -- 172.21.15.114:0/902736346 shutdown_connections 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/902736346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90158a50 0x7fcd900c98b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 -- 172.21.15.114:0/902736346 >> 172.21.15.114:0/902736346 conn(0x7fcd900b57b0 msgr2=0x7fcd900b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 -- 172.21.15.114:0/902736346 shutdown_connections 2023-11-24T06:25:10.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.536+0000 7fcd97d8f700 1 -- 172.21.15.114:0/902736346 wait complete. 2023-11-24T06:25:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.537+0000 7fcd97d8f700 1 Processor -- start 2023-11-24T06:25:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.537+0000 7fcd97d8f700 1 -- start start 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd90158670 0x7fcd900cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 0x7fcd9014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90159ff0 con 0x7fcd90162a30 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900cb160 con 0x7fcd90158a50 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90157850 con 0x7fcd90158670 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39536/0 (socket says 172.21.15.114:39536) 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 -- 172.21.15.114:0/2143236979 learned_addr learned my addr 172.21.15.114:0/2143236979 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd95b2b700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd90158670 0x7fcd900cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9532a700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 0x7fcd9014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:10.544 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 -- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd90158670 msgr2=0x7fcd900cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd90158670 0x7fcd900cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 -- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 msgr2=0x7fcd9014d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 0x7fcd9014d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 -- 172.21.15.114:0/2143236979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0094a0 con 0x7fcd90162a30 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9532a700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 0x7fcd9014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd9632c700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800cb90 tx=0x7fcd8800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800ab30 con 0x7fcd90162a30 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8800ac90 con 0x7fcd90162a30 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd90159790 con 0x7fcd90162a30 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.538+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880070d0 con 0x7fcd90162a30 2023-11-24T06:25:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.539+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd90159d10 con 0x7fcd90162a30 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.539+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd90067f60 con 0x7fcd90162a30 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.540+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd88007230 con 0x7fcd90162a30 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.540+0000 7fcd867fc700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.540+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcd88088e50 con 0x7fcd90162a30 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.540+0000 7fcd95b2b700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:10.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.541+0000 7fcd95b2b700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 0x7fcd7406d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd80000f60 tx=0x7fcd80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:10.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.545+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd88054200 con 0x7fcd90162a30 2023-11-24T06:25:10.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-24T06:25:10.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-24T06:25:10.568 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-24T06:25:10.568 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-24T06:25:10.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:10.751+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcd900b0680 con 0x7fcd90162a30 2023-11-24T06:25:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.336+0000 7fcd867fc700 1 -- 172.21.15.114:0/2143236979 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd88018020 con 0x7fcd90162a30 2023-11-24T06:25:11.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.338+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 msgr2=0x7fcd7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.338+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 0x7fcd7406d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd80000f60 tx=0x7fcd80009040 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.338+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 msgr2=0x7fcd9015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.338+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800cb90 tx=0x7fcd8800cf50 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 shutdown_connections 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd7406aed0 0x7fcd7406d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd90162a30 0x7fcd9015b4b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd90158670 0x7fcd900cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 --2- 172.21.15.114:0/2143236979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd90158a50 0x7fcd9014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 >> 172.21.15.114:0/2143236979 conn(0x7fcd900b57b0 msgr2=0x7fcd900cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 shutdown_connections 2023-11-24T06:25:11.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.339+0000 7fcd97d8f700 1 -- 172.21.15.114:0/2143236979 wait complete. 2023-11-24T06:25:11.344 INFO:teuthology.orchestra.run.smithi114.stderr: removed all blocklist entries 2023-11-24T06:25:11.358 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:25:11.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.480+0000 7f70e7885700 1 Processor -- start 2023-11-24T06:25:11.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.481+0000 7f70e7885700 1 -- start start 2023-11-24T06:25:11.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.481+0000 7f70e7885700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70e01446a0 0x7f70e00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e7885700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70e00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e7885700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e00af950 con 0x7f70e00adf00 2023-11-24T06:25:11.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e7885700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e0144a80 con 0x7f70e01446a0 2023-11-24T06:25:11.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e7885700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00afc30 con 0x7f70e006c710 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e4e20700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f70e00adf00 0x7f70e006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55248/0 (socket says 172.21.15.114:55248) 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e4e20700 1 -- 172.21.15.114:0/597443096 learned_addr learned my addr 172.21.15.114:0/597443096 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e5e22700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70e00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340065028 0 0) 0x7f70e00af950 con 0x7f70e00adf00 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70cc0035b0 con 0x7f70e00adf00 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e5621700 1 --2- 172.21.15.114:0/597443096 >> v2:172.21.15.172:3300/0 conn(0x7f70e01446a0 0x7f70e00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2100464558 0 0) 0x7f70cc0035b0 con 0x7f70e00adf00 2023-11-24T06:25:11.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 msgr2=0x7f70e00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70e00b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 >> v2:172.21.15.172:3300/0 conn(0x7f70e01446a0 msgr2=0x7f70e00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 --2- 172.21.15.114:0/597443096 >> v2:172.21.15.172:3300/0 conn(0x7f70e01446a0 0x7f70e00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00aff10 con 0x7f70e00adf00 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f70dc001670 con 0x7f70e00adf00 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 >> v1:172.21.15.114:6789/0 conn(0x7f70e00adf00 legacy=0x7f70e006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70cc003a00 0x7f70cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70cc004d90 0x7f70cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0144a80 con 0x7f70cc003a00 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00afc30 con 0x7f70cc004d90 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc00ba10 con 0x7f70e006c710 2023-11-24T06:25:11.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e5621700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70cc003a00 0x7f70cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e4e20700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.482+0000 7f70e5e22700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70cc004d90 0x7f70cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 -- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 msgr2=0x7f70cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e006c710 0x7f70cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 -- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70cc004d90 msgr2=0x7f70cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70cc004d90 0x7f70cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 -- 172.21.15.114:0/597443096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc0035b0 con 0x7f70cc003a00 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5621700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70cc003a00 0x7f70cc003e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009b60 tx=0x7f70cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a9b7a3bc0a76e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000e6c0 con 0x7f70cc003a00 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d000e820 con 0x7f70cc003a00 2023-11-24T06:25:11.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70d7fff700 1 -- 172.21.15.114:0/597443096 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000eb90 con 0x7f70cc003a00 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e7885700 1 -- 172.21.15.114:0/597443096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70cc003a00 msgr2=0x7f70cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e7885700 1 --2- 172.21.15.114:0/597443096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70cc003a00 0x7f70cc003e20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009b60 tx=0x7f70cc005880 comp rx=0 tx=0).stop 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e5e22700 1 -- 172.21.15.114:0/597443096 reap_dead start 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e7885700 1 -- 172.21.15.114:0/597443096 shutdown_connections 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.483+0000 7f70e7885700 1 -- 172.21.15.114:0/597443096 >> 172.21.15.114:0/597443096 conn(0x7f70e0071b90 msgr2=0x7f70e0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.484+0000 7f70e7885700 1 -- 172.21.15.114:0/597443096 shutdown_connections 2023-11-24T06:25:11.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.484+0000 7f70e7885700 1 -- 172.21.15.114:0/597443096 wait complete. 2023-11-24T06:25:11.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.485+0000 7f70e7885700 1 Processor -- start 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.485+0000 7f70e7885700 1 -- start start 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70cc003a00 0x7f70e01410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e0143370 0x7f70e01f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01f7680 con 0x7f70e0142ef0 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e013edb0 con 0x7f70e0143370 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e7885700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e013f090 con 0x7f70cc003a00 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e4e20700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e4e20700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39554/0 (socket says 172.21.15.114:39554) 2023-11-24T06:25:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e4e20700 1 -- 172.21.15.114:0/2087328927 learned_addr learned my addr 172.21.15.114:0/2087328927 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e5621700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70cc003a00 0x7f70e01410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.486+0000 7f70e5e22700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e0143370 0x7f70e01f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 -- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70cc003a00 msgr2=0x7f70e01410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70cc003a00 0x7f70e01410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 -- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e0143370 msgr2=0x7f70e01f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e0143370 0x7f70e01f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 -- 172.21.15.114:0/2087328927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e013f370 con 0x7f70e0142ef0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e4e20700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007310 tx=0x7f70dc007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011580 con 0x7f70e0142ef0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0116e0 con 0x7f70e0142ef0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e013f650 con 0x7f70e0142ef0 2023-11-24T06:25:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc011a90 con 0x7f70e0142ef0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.487+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e00b0bf0 con 0x7f70e0142ef0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.488+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00753f0 con 0x7f70e0142ef0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.488+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70dc011bf0 con 0x7f70e0142ef0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.489+0000 7f70d5ffb700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 0x7f70c406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.489+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f70dc0891c0 con 0x7f70e0142ef0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.489+0000 7f70e5621700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 0x7f70c406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.489+0000 7f70e5621700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 0x7f70c406d380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f70d00090b0 tx=0x7f70d001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.493+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70dc054650 con 0x7f70e0142ef0 2023-11-24T06:25:11.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.761+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f70e013fb70 con 0x7f70e0142ef0 2023-11-24T06:25:11.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.761+0000 7f70d5ffb700 1 -- 172.21.15.114:0/2087328927 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f70e013fb70 con 0x7f70e0142ef0 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "auth_dump": [ 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.a", 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBloXHaIBAA40BWWVcYcVbX+kM3wd4srw==", 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:11.765 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.b", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlRJM0LRAAToQLwjepMjKSY6cakPeFJw==", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.c", 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBllY8dKBAAtxCS400KLv4rt6WIzc3a2g==", 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.767 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.d", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlwqqDNBAARSrqgGNEALaFrG0ZFStYgg==", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.0", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBZQWBlQ7/gFxAA603E5V+l/7dkugKMUNQV3A==", 2023-11-24T06:25:11.768 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.1", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBcQWBlnHtEFxAAr9h0RnKScdwcnLc2O9Pp4A==", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.2", 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBfQWBlmYVaGBAARyCei1YbVxxzy00k0W/Z6w==", 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.770 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.3", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBjQWBlVqdEABAAyD3s0xW8jXecRwdgtq8AwQ==", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.4", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBpQWBlyLk4BxAAauMGnn0akTG5zKYKcy9Vmw==", 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.771 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.5", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBsQWBlQjG4BhAAuv1Ja24nPcZ+SmnV8yuHgg==", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.6", 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBvQWBl4tb6BRAAcoydr7CybV8Ckv++5LFS+w==", 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.7", 2023-11-24T06:25:11.773 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQByQWBlIUmuBRAAO0eSyPnvmpUft7iB4vQ6dg==", 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.774 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.0", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g==", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.1", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlSFhWBhAA13v9kza/+snKVCmwnfK4eA==", 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.775 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.2", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlKtb6AhAAoaQhrX2DZgvNW0rcCB1iVg==", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:11.776 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.3", 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlwshBDBAAkMjk+usVBY7TYehxOhJdDw==", 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.admin", 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlz1e2AhAAHaN4NjkwUY+J8W2nidj3Gg==", 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBlvUoeBRAA/plNPRuXiSeMiB6q4A7POQ==", 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBlS1UeBRAAJSmMzwWy8hO+3Md3PYcuIQ==", 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBlBV0eBRAAiuEwzjU4G/K2E+WHOss+Gw==", 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBlPmQeBRAALvunYZN3qTj9y5LGGgN9lw==", 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:25:11.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBltWseBRAAKJbktq/6Fj8nqmXZfM1e1g==", 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQB4QWBlcnMeBRAAcLq/ArfGJG4umQcvjXxR0A==", 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.781 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.x", 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlT3B0HRAANmHJk8Er7FHsAeQJuAnUKg==", 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:11.782 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.y", 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlcY5dGBAAZ7ISSGzT/OCSkRVGTBwkvA==", 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2023-11-24T06:25:11.783 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:25:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 msgr2=0x7f70c406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 0x7f70c406d380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f70d00090b0 tx=0x7f70d001f040 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 msgr2=0x7f70e01f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007310 tx=0x7f70dc007620 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 shutdown_connections 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f70c406aec0 0x7f70c406d380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e0143370 0x7f70e01f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70cc003a00 0x7f70e01410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 --2- 172.21.15.114:0/2087328927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e0142ef0 0x7f70e01f6c00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 >> 172.21.15.114:0/2087328927 conn(0x7f70e0071b90 msgr2=0x7f70e006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 shutdown_connections 2023-11-24T06:25:11.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.763+0000 7f70e7885700 1 -- 172.21.15.114:0/2087328927 wait complete. 2023-11-24T06:25:11.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:25:11.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.903+0000 7fecc3733700 1 Processor -- start 2023-11-24T06:25:11.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.903+0000 7fecc3733700 1 -- start start 2023-11-24T06:25:11.913 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc3733700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fecbc06c710 0x7fecbc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.913 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc3733700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0ae430 con 0x7fecbc0af4d0 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc3733700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0ae710 con 0x7fecbc15e080 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc3733700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0af0b0 con 0x7fecbc06c710 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc1cd0700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fecbc0af4d0 0x7fecbc06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55272/0 (socket says 172.21.15.114:55272) 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc1cd0700 1 -- 172.21.15.114:0/4125461731 learned_addr learned my addr 172.21.15.114:0/4125461731 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155463313 0 0) 0x7fecbc0ae430 con 0x7fecbc0af4d0 2023-11-24T06:25:11.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feca80035b0 con 0x7fecbc0af4d0 2023-11-24T06:25:11.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecc0cce700 1 --2- 172.21.15.114:0/4125461731 >> v2:172.21.15.172:3300/0 conn(0x7fecbc06c710 0x7fecbc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147555673 0 0) 0x7feca80035b0 con 0x7fecbc0af4d0 2023-11-24T06:25:11.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 >> v2:172.21.15.172:3300/0 conn(0x7fecbc06c710 msgr2=0x7fecbc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 --2- 172.21.15.114:0/4125461731 >> v2:172.21.15.172:3300/0 conn(0x7fecbc06c710 0x7fecbc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 >> v1:172.21.15.172:6789/0 conn(0x7fecbc15e080 legacy=0x7fecbc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc0ae9f0 con 0x7fecbc0af4d0 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fecb40019f0 con 0x7fecbc0af4d0 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 >> v1:172.21.15.114:6789/0 conn(0x7fecbc0af4d0 legacy=0x7fecbc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.904+0000 7fecb3fff700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7feca8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8004850 0x7feca8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.916 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca8005210 0x7feca800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0af0b0 con 0x7feca8003a00 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca800c0b0 con 0x7feca8004850 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca800c270 con 0x7feca8005210 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7feca8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc0cce700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8004850 0x7feca8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.917 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc1cd0700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca8005210 0x7feca800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 -- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca8005210 msgr2=0x7feca800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca8005210 0x7feca800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 -- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8004850 msgr2=0x7feca8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8004850 0x7feca8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc1cd0700 1 -- 172.21.15.114:0/4125461731 reap_dead start 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 -- 172.21.15.114:0/4125461731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80035b0 con 0x7feca8003a00 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc0cce700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8004850 0x7feca8004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecc14cf700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7feca8003e20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecb80017c0 tx=0x7feca8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a5d4b1c5c7f402 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80052d0 con 0x7feca8003a00 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8005430 con 0x7feca8003a00 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.905+0000 7fecb3fff700 1 -- 172.21.15.114:0/4125461731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80038c0 con 0x7feca8003a00 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 -- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 msgr2=0x7feca8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7feca8003e20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecb80017c0 tx=0x7feca8005880 comp rx=0 tx=0).stop 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 -- 172.21.15.114:0/4125461731 shutdown_connections 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 --2- 172.21.15.114:0/4125461731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7feca8003e20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 -- 172.21.15.114:0/4125461731 >> 172.21.15.114:0/4125461731 conn(0x7fecbc071b90 msgr2=0x7fecbc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 -- 172.21.15.114:0/4125461731 shutdown_connections 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.906+0000 7fecc3733700 1 -- 172.21.15.114:0/4125461731 wait complete. 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.907+0000 7fecc3733700 1 Processor -- start 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.907+0000 7fecc3733700 1 -- start start 2023-11-24T06:25:11.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 0x7fecbc14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 0x7fecbc0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0cb5c0 con 0x7feca8003a00 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0c98c0 con 0x7fecbc0c8f00 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc14f260 con 0x7fecbc14e7a0 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39572/0 (socket says 172.21.15.114:39572) 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 -- 172.21.15.114:0/3184690415 learned_addr learned my addr 172.21.15.114:0/3184690415 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc1cd0700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 0x7fecbc0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc0cce700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 0x7fecbc14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 -- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 msgr2=0x7fecbc14ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 0x7fecbc14ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 -- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 msgr2=0x7fecbc0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 0x7fecbc0c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 -- 172.21.15.114:0/3184690415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc14f540 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc0cce700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 0x7fecbc14ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc1cd0700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 0x7fecbc0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc14cf700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feca800c080 tx=0x7fecb8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8001290 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8007040 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8007410 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.908+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc0cb250 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.909+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc0c6770 con 0x7feca8003a00 2023-11-24T06:25:11.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.910+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc0753f0 con 0x7feca8003a00 2023-11-24T06:25:11.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.910+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecb8007570 con 0x7feca8003a00 2023-11-24T06:25:11.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.910+0000 7fecb1ffb700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 0x7feca006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:11.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.911+0000 7fecc0cce700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 0x7feca006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:11.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.911+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fecb8089c00 con 0x7feca8003a00 2023-11-24T06:25:11.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.911+0000 7fecc0cce700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 0x7feca006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0691b0 tx=0x7fecac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:11.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:11.915+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecb8090080 con 0x7feca8003a00 2023-11-24T06:25:12.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.183+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fecbc14f730 con 0x7feca8003a00 2023-11-24T06:25:12.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.187+0000 7fecb1ffb700 1 -- 172.21.15.114:0/3184690415 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fecbc14f730 con 0x7feca8003a00 2023-11-24T06:25:12.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 msgr2=0x7feca006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 0x7feca006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0691b0 tx=0x7fecac008040 comp rx=0 tx=0).stop 2023-11-24T06:25:12.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 msgr2=0x7fecbc14c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feca800c080 tx=0x7fecb8001120 comp rx=0 tx=0).stop 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 shutdown_connections 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feca006aed0 0x7feca006d390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecbc0c8f00 0x7fecbc0c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feca8003a00 0x7fecbc14c060 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 --2- 172.21.15.114:0/3184690415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecbc14e7a0 0x7fecbc14ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 >> 172.21.15.114:0/3184690415 conn(0x7fecbc071b90 msgr2=0x7fecbc06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:12.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 shutdown_connections 2023-11-24T06:25:12.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.189+0000 7fecc3733700 1 -- 172.21.15.114:0/3184690415 wait complete. 2023-11-24T06:25:12.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:25:12.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.322+0000 7fc66f8b3700 1 Processor -- start 2023-11-24T06:25:12.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 -- start start 2023-11-24T06:25:12.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66815e080 0x7fc6680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6680af4d0 0x7fc6680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806c000 0x7fc66806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680af0b0 con 0x7fc66815e080 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66806c9f0 con 0x7fc6680af4d0 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66f8b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b05a0 con 0x7fc66806c000 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.323+0000 7fc66d64f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66815e080 0x7fc6680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66815e080 0x7fc6680b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39580/0 (socket says 172.21.15.114:39580) 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 -- 172.21.15.114:0/1615972269 learned_addr learned my addr 172.21.15.114:0/1615972269 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66de50700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806c000 0x7fc66806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66ce4e700 1 --2- 172.21.15.114:0/1615972269 >> v2:172.21.15.172:3300/0 conn(0x7fc6680af4d0 0x7fc6680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 -- 172.21.15.114:0/1615972269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806c000 msgr2=0x7fc66806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806c000 0x7fc66806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 -- 172.21.15.114:0/1615972269 >> v2:172.21.15.172:3300/0 conn(0x7fc6680af4d0 msgr2=0x7fc6680afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 --2- 172.21.15.114:0/1615972269 >> v2:172.21.15.172:3300/0 conn(0x7fc6680af4d0 0x7fc6680afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 -- 172.21.15.114:0/1615972269 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680ae430 con 0x7fc66815e080 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc66d64f700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66815e080 0x7fc6680b5050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc66806e4e0 tx=0x7fc658009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3addcb1583097465 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc65ffff700 1 -- 172.21.15.114:0/1615972269 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800b870 con 0x7fc66815e080 2023-11-24T06:25:12.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc65ffff700 1 -- 172.21.15.114:0/1615972269 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65800b9d0 con 0x7fc66815e080 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.324+0000 7fc65ffff700 1 -- 172.21.15.114:0/1615972269 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800bd60 con 0x7fc66815e080 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 -- 172.21.15.114:0/1615972269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66815e080 msgr2=0x7fc6680b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66815e080 0x7fc6680b5050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc66806e4e0 tx=0x7fc658009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 -- 172.21.15.114:0/1615972269 shutdown_connections 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66806c000 0x7fc66806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/1615972269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66815e080 0x7fc6680b5050 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/1615972269 >> v2:172.21.15.172:3300/0 conn(0x7fc6680af4d0 0x7fc6680afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 -- 172.21.15.114:0/1615972269 >> 172.21.15.114:0/1615972269 conn(0x7fc668071b90 msgr2=0x7fc668071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 -- 172.21.15.114:0/1615972269 shutdown_connections 2023-11-24T06:25:12.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.325+0000 7fc66f8b3700 1 -- 172.21.15.114:0/1615972269 wait complete. 2023-11-24T06:25:12.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.326+0000 7fc66f8b3700 1 Processor -- start 2023-11-24T06:25:12.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.326+0000 7fc66f8b3700 1 -- start start 2023-11-24T06:25:12.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680af4d0 0x7fc66815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66815e080 0x7fc668154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668156840 con 0x7fc66806c000 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668151e80 con 0x7fc66815e080 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66f8b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc668157340 con 0x7fc6680af4d0 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66d64f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66d64f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39588/0 (socket says 172.21.15.114:39588) 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66d64f700 1 -- 172.21.15.114:0/3049277923 learned_addr learned my addr 172.21.15.114:0/3049277923 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:12.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66ce4e700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680af4d0 0x7fc66815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.327+0000 7fc66de50700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66815e080 0x7fc668154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 -- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680af4d0 msgr2=0x7fc66815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680af4d0 0x7fc66815f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 -- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66815e080 msgr2=0x7fc668154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66815e080 0x7fc668154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 -- 172.21.15.114:0/3049277923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658009550 con 0x7fc66806c000 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66d64f700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc658000c00 tx=0x7fc658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658004520 con 0x7fc66806c000 2023-11-24T06:25:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65801e070 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc668157620 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580038c0 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.328+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc668157be0 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.329+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc668157da0 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.330+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc658003a20 con 0x7fc66806c000 2023-11-24T06:25:12.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.330+0000 7fc65dffb700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 0x7fc64c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.330+0000 7fc66ce4e700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 0x7fc64c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.330+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc658020070 con 0x7fc66806c000 2023-11-24T06:25:12.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.331+0000 7fc66ce4e700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 0x7fc64c06d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6680b2290 tx=0x7fc664008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.335+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc658055af0 con 0x7fc66806c000 2023-11-24T06:25:12.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.602+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc6680753f0 con 0x7fc66806c000 2023-11-24T06:25:12.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.604+0000 7fc65dffb700 1 -- 172.21.15.114:0/3049277923 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc6580150a0 con 0x7fc66806c000 2023-11-24T06:25:12.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 msgr2=0x7fc64c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 0x7fc64c06d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6680b2290 tx=0x7fc664008040 comp rx=0 tx=0).stop 2023-11-24T06:25:12.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 msgr2=0x7fc668151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc658000c00 tx=0x7fc658004750 comp rx=0 tx=0).stop 2023-11-24T06:25:12.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 shutdown_connections 2023-11-24T06:25:12.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc64c06aed0 0x7fc64c06d390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.612 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66815e080 0x7fc668154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66806c000 0x7fc668151940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 --2- 172.21.15.114:0/3049277923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6680af4d0 0x7fc66815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 >> 172.21.15.114:0/3049277923 conn(0x7fc668071b90 msgr2=0x7fc668073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:12.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 shutdown_connections 2023-11-24T06:25:12.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.607+0000 7fc66f8b3700 1 -- 172.21.15.114:0/3049277923 wait complete. 2023-11-24T06:25:12.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:25:12.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 Processor -- start 2023-11-24T06:25:12.743 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 -- start start 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69e40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e40ae8d0 con 0x7f69e40b44b0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e415e790 con 0x7f69e4071a50 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.740+0000 7f69eb020700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e40b4ae0 con 0x7f69e406eca0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e3fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f69e40b44b0 0x7f69e4071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55308/0 (socket says 172.21.15.114:55308) 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e3fff700 1 -- 172.21.15.114:0/1498636232 learned_addr learned my addr 172.21.15.114:0/1498636232 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e8dbc700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69e40b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277010979 0 0) 0x7f69e40ae8d0 con 0x7f69e40b44b0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d40035b0 con 0x7f69e40b44b0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2736242039 0 0) 0x7f69d40035b0 con 0x7f69e40b44b0 2023-11-24T06:25:12.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 msgr2=0x7f69e40b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69e40b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 >> v1:172.21.15.172:6789/0 conn(0x7f69e4071a50 legacy=0x7f69e40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40b4dc0 con 0x7f69e40b44b0 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f69d0001670 con 0x7f69e40b44b0 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 >> v1:172.21.15.114:6789/0 conn(0x7f69e40b44b0 legacy=0x7f69e4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69d4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d4003f70 0x7f69d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e40b4ae0 con 0x7f69d4004d90 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d400ba70 con 0x7f69d4003f70 2023-11-24T06:25:12.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d400bc30 con 0x7f69e406eca0 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e95bd700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e8dbc700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69d4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.741+0000 7f69e3fff700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d4003f70 0x7f69d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 -- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 msgr2=0x7f69d4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e406eca0 0x7f69d4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 -- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d4003f70 msgr2=0x7f69d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d4003f70 0x7f69d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 -- 172.21.15.114:0/1498636232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d40035b0 con 0x7f69d4004d90 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69d4005210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c290 tx=0x7f69d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c89100022d0a9a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc001e70 con 0x7f69d4004d90 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc00ce50 con 0x7f69d4004d90 2023-11-24T06:25:12.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e37fe700 1 -- 172.21.15.114:0/1498636232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc012660 con 0x7f69d4004d90 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 -- 172.21.15.114:0/1498636232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 msgr2=0x7f69d4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 --2- 172.21.15.114:0/1498636232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69d4005210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c290 tx=0x7f69d4005880 comp rx=0 tx=0).stop 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69e95bd700 1 -- 172.21.15.114:0/1498636232 reap_dead start 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 -- 172.21.15.114:0/1498636232 shutdown_connections 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 -- 172.21.15.114:0/1498636232 >> 172.21.15.114:0/1498636232 conn(0x7f69e40743b0 msgr2=0x7f69e406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 -- 172.21.15.114:0/1498636232 shutdown_connections 2023-11-24T06:25:12.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.742+0000 7f69eb020700 1 -- 172.21.15.114:0/1498636232 wait complete. 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.744+0000 7f69eb020700 1 Processor -- start 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.744+0000 7f69eb020700 1 -- start start 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e41f3950 0x7f69e41f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e4153570 0x7f69e41539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4153150 con 0x7f69d4004d90 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4153f30 con 0x7f69e4153570 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69eb020700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41524d0 con 0x7f69e41f3950 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39610/0 (socket says 172.21.15.114:39610) 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 -- 172.21.15.114:0/3054365834 learned_addr learned my addr 172.21.15.114:0/3054365834 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 -- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e41f3950 msgr2=0x7f69e41f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e41f3950 0x7f69e41f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 -- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e4153570 msgr2=0x7f69e41539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e4153570 0x7f69e41539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.745+0000 7f69e8dbc700 1 -- 172.21.15.114:0/3054365834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e41527b0 con 0x7f69d4004d90 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69e8dbc700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69d8011aa0 tx=0x7f69d8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800f900 con 0x7f69d4004d90 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d800fa60 con 0x7f69d4004d90 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e4152a90 con 0x7f69d4004d90 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800fe10 con 0x7f69d4004d90 2023-11-24T06:25:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.746+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e40b2840 con 0x7f69d4004d90 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.747+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e40753f0 con 0x7f69d4004d90 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.747+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69d8014440 con 0x7f69d4004d90 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.747+0000 7f69e17fa700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 0x7f69c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.748+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69d8089910 con 0x7f69d4004d90 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.748+0000 7f69e3fff700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 0x7f69c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.748+0000 7f69e3fff700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 0x7f69c806d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69d0001050 tx=0x7f69d0001b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:12.752+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69d8054e60 con 0x7f69d4004d90 2023-11-24T06:25:13.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.025+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f69e41533c0 con 0x7f69d4004d90 2023-11-24T06:25:13.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.027+0000 7f69e17fa700 1 -- 172.21.15.114:0/3054365834 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f69d8058480 con 0x7f69d4004d90 2023-11-24T06:25:13.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.029+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 msgr2=0x7f69c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.029+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 0x7f69c806d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69d0001050 tx=0x7f69d0001b40 comp rx=0 tx=0).stop 2023-11-24T06:25:13.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.029+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 msgr2=0x7f69e40b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.029+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69d8011aa0 tx=0x7f69d8011e60 comp rx=0 tx=0).stop 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 shutdown_connections 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f69c806aed0 0x7f69c806d390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e4153570 0x7f69e41539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d4004d90 0x7f69e40b2300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 --2- 172.21.15.114:0/3054365834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e41f3950 0x7f69e41f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 >> 172.21.15.114:0/3054365834 conn(0x7f69e40743b0 msgr2=0x7f69e4157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 shutdown_connections 2023-11-24T06:25:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.030+0000 7f69eb020700 1 -- 172.21.15.114:0/3054365834 wait complete. 2023-11-24T06:25:13.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:25:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.163+0000 7fa3fdc67700 1 Processor -- start 2023-11-24T06:25:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.163+0000 7fa3fdc67700 1 -- start start 2023-11-24T06:25:13.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3fdc67700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa3f80d39d0 0x7fa3f80b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3fdc67700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80d2930 con 0x7fa3f815e140 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3fdc67700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f80d2c10 con 0x7fa3f80d45d0 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3fdc67700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f815e520 con 0x7fa3f80d39d0 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f7fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa3f80d39d0 0x7fa3f80b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f77fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa3f815e140 0x7fa3f80d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55326/0 (socket says 172.21.15.114:55326) 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f77fe700 1 -- 172.21.15.114:0/3772142027 learned_addr learned my addr 172.21.15.114:0/3772142027 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f7fff700 1 -- 172.21.15.114:0/3772142027 >> v1:172.21.15.172:6789/0 conn(0x7fa3f80d45d0 legacy=0x7fa3f80b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f7fff700 1 -- 172.21.15.114:0/3772142027 >> v1:172.21.15.114:6789/0 conn(0x7fa3f815e140 legacy=0x7fa3f80d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f67fc700 1 -- 172.21.15.114:0/3772142027 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984263869 0 0) 0x7fa3f80d2930 con 0x7fa3f815e140 2023-11-24T06:25:13.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f7fff700 1 -- 172.21.15.114:0/3772142027 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80d2ef0 con 0x7fa3f80d39d0 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f7fff700 1 --2- 172.21.15.114:0/3772142027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f80d39d0 0x7fa3f80b77e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009d90 tx=0x7fa3f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2de0d49d6eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f67fc700 1 -- 172.21.15.114:0/3772142027 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000e070 con 0x7fa3f80d39d0 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f67fc700 1 -- 172.21.15.114:0/3772142027 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f000bbd0 con 0x7fa3f80d39d0 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.164+0000 7fa3f67fc700 1 -- 172.21.15.114:0/3772142027 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0012430 con 0x7fa3f80d39d0 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 -- 172.21.15.114:0/3772142027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f80d39d0 msgr2=0x7fa3f80b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/3772142027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f80d39d0 0x7fa3f80b77e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009d90 tx=0x7fa3f000b330 comp rx=0 tx=0).stop 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 -- 172.21.15.114:0/3772142027 shutdown_connections 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/3772142027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f80d39d0 0x7fa3f80b77e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 -- 172.21.15.114:0/3772142027 >> 172.21.15.114:0/3772142027 conn(0x7fa3f806b410 msgr2=0x7fa3f806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 -- 172.21.15.114:0/3772142027 shutdown_connections 2023-11-24T06:25:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.165+0000 7fa3fdc67700 1 -- 172.21.15.114:0/3772142027 wait complete. 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.166+0000 7fa3fdc67700 1 Processor -- start 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.166+0000 7fa3fdc67700 1 -- start start 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f80d39d0 0x7fa3f814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f80d45d0 0x7fa3f80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80c7db0 con 0x7fa3f815e140 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f814f950 con 0x7fa3f80d45d0 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3fdc67700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80cc9b0 con 0x7fa3f80d39d0 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39628/0 (socket says 172.21.15.114:39628) 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 -- 172.21.15.114:0/735901621 learned_addr learned my addr 172.21.15.114:0/735901621 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f77fe700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f80d39d0 0x7fa3f814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f6ffd700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f80d45d0 0x7fa3f80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 -- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f80d39d0 msgr2=0x7fa3f814f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f80d39d0 0x7fa3f814f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 -- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f80d45d0 msgr2=0x7fa3f80ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f80d45d0 0x7fa3f80ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.167+0000 7fa3f7fff700 1 -- 172.21.15.114:0/735901621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009a70 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3f7fff700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0014040 tx=0x7fa3f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001b810 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0020070 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80c7760 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f001bbb0 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.168+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f81f4990 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.169+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80c75c0 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.169+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3f001bd10 con 0x7fa3f815e140 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.170+0000 7fa3dffff700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 0x7fa3e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.170+0000 7fa3f77fe700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 0x7fa3e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.170+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3f0022070 con 0x7fa3f815e140 2023-11-24T06:25:13.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.170+0000 7fa3f77fe700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 0x7fa3e406d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000f60 tx=0x7fa3e8007db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.174+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3f80c75c0 con 0x7fa3f815e140 2023-11-24T06:25:13.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.443+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa3f80c75c0 con 0x7fa3f815e140 2023-11-24T06:25:13.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.445+0000 7fa3dffff700 1 -- 172.21.15.114:0/735901621 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa3f80c75c0 con 0x7fa3f815e140 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 msgr2=0x7fa3e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 0x7fa3e406d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000f60 tx=0x7fa3e8007db0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 msgr2=0x7fa3f80cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0014040 tx=0x7fa3f0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 shutdown_connections 2023-11-24T06:25:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa3e406aed0 0x7fa3e406d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f815e140 0x7fa3f80cd440 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3f80d39d0 0x7fa3f814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 --2- 172.21.15.114:0/735901621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3f80d45d0 0x7fa3f80ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.447+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 >> 172.21.15.114:0/735901621 conn(0x7fa3f806b410 msgr2=0x7fa3f815e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.448+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 shutdown_connections 2023-11-24T06:25:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.448+0000 7fa3fdc67700 1 -- 172.21.15.114:0/735901621 wait complete. 2023-11-24T06:25:13.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:25:13.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.580+0000 7f0926b71700 1 Processor -- start 2023-11-24T06:25:13.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.580+0000 7f0926b71700 1 -- start start 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f092006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09200b49a0 0x7f092006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 0x7f09200d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092015c4b0 con 0x7f09200b49a0 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092015c790 con 0x7f09200cff20 2023-11-24T06:25:13.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f0926b71700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092015ca70 con 0x7f0920152cc0 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f091ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09200b49a0 0x7f092006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f091ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09200b49a0 0x7f092006c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39632/0 (socket says 172.21.15.114:39632) 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.581+0000 7f091ffff700 1 -- 172.21.15.114:0/2139232119 learned_addr learned my addr 172.21.15.114:0/2139232119 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f092490d700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f092006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f092510e700 1 --2- 172.21.15.114:0/2139232119 >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 0x7f09200d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 -- 172.21.15.114:0/2139232119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 msgr2=0x7f092006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f092006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 -- 172.21.15.114:0/2139232119 >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 msgr2=0x7f09200d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 --2- 172.21.15.114:0/2139232119 >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 0x7f09200d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 -- 172.21.15.114:0/2139232119 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092015cd50 con 0x7f09200b49a0 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f092490d700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f092006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:13.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f092510e700 1 --2- 172.21.15.114:0/2139232119 >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 0x7f09200d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091ffff700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200b49a0 0x7f092006c540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0920159e50 tx=0x7f090c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215e0651c26a2d95 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091f7fe700 1 -- 172.21.15.114:0/2139232119 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00b870 con 0x7f09200b49a0 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091f7fe700 1 -- 172.21.15.114:0/2139232119 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00b9d0 con 0x7f09200b49a0 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f091f7fe700 1 -- 172.21.15.114:0/2139232119 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00bd60 con 0x7f09200b49a0 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 -- 172.21.15.114:0/2139232119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200b49a0 msgr2=0x7f092006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200b49a0 0x7f092006c540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0920159e50 tx=0x7f090c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 -- 172.21.15.114:0/2139232119 shutdown_connections 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f092006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 --2- 172.21.15.114:0/2139232119 >> v2:172.21.15.172:3300/0 conn(0x7f09200cff20 0x7f09200d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.582+0000 7f0926b71700 1 --2- 172.21.15.114:0/2139232119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200b49a0 0x7f092006c540 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.583+0000 7f0926b71700 1 -- 172.21.15.114:0/2139232119 >> 172.21.15.114:0/2139232119 conn(0x7f0920071b90 msgr2=0x7f0920071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:13.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.583+0000 7f0926b71700 1 -- 172.21.15.114:0/2139232119 shutdown_connections 2023-11-24T06:25:13.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.583+0000 7f0926b71700 1 -- 172.21.15.114:0/2139232119 wait complete. 2023-11-24T06:25:13.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.584+0000 7f0926b71700 1 Processor -- start 2023-11-24T06:25:13.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.584+0000 7f0926b71700 1 -- start start 2023-11-24T06:25:13.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09200b49a0 0x7f0920148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f09200adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0920148b40 con 0x7f09200cff20 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09201f6090 con 0x7f09200b49a0 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09201544c0 con 0x7f0920152cc0 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39640/0 (socket says 172.21.15.114:39640) 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 -- 172.21.15.114:0/2609837131 learned_addr learned my addr 172.21.15.114:0/2609837131 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f092490d700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09200b49a0 0x7f0920148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f092510e700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f09200adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 -- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 msgr2=0x7f09200adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f09200adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 -- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09200b49a0 msgr2=0x7f0920148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09200b49a0 0x7f0920148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 -- 172.21.15.114:0/2609837131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009550 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091ffff700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09200728e0 tx=0x7f090c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c004310 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c01f070 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09201f5ad0 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c0036a0 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.585+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0920154150 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.586+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09201547a0 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.587+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f090c021030 con 0x7f09200cff20 2023-11-24T06:25:13.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.587+0000 7f091d7fa700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 0x7f090406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:13.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.587+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f090c08a7f0 con 0x7f09200cff20 2023-11-24T06:25:13.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.587+0000 7f092490d700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 0x7f090406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:13.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.587+0000 7f092490d700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 0x7f090406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f0914008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:13.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.591+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f090c055da0 con 0x7f09200cff20 2023-11-24T06:25:13.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.862+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f09200753f0 con 0x7f09200cff20 2023-11-24T06:25:13.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.864+0000 7f091d7fa700 1 -- 172.21.15.114:0/2609837131 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f090c0150a0 con 0x7f09200cff20 2023-11-24T06:25:13.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 msgr2=0x7f090406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 0x7f090406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0914006fd0 tx=0x7f0914008040 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 msgr2=0x7f09201f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09200728e0 tx=0x7f090c004750 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 shutdown_connections 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f090406aed0 0x7f090406d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0920152cc0 0x7f09200adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09200b49a0 0x7f0920148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 --2- 172.21.15.114:0/2609837131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09200cff20 0x7f09201f5110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 >> 172.21.15.114:0/2609837131 conn(0x7f0920071b90 msgr2=0x7f0920074800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:13.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 shutdown_connections 2023-11-24T06:25:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:13.866+0000 7f0926b71700 1 -- 172.21.15.114:0/2609837131 wait complete. 2023-11-24T06:25:13.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:25:14.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.000+0000 7fdbe5be6700 1 Processor -- start 2023-11-24T06:25:14.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.000+0000 7fdbe5be6700 1 -- start start 2023-11-24T06:25:14.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbe5be6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbe00b3370 0x7fdbe01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbe5be6700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00b22d0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbe5be6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00b25b0 con 0x7fdbe00d1c70 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbe5be6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe00b2f50 con 0x7fdbe00b3370 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbe00b3370 0x7fdbe01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbe00b3370 0x7fdbe01604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39648/0 (socket says 172.21.15.114:39648) 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 -- 172.21.15.114:0/1183028636 learned_addr learned my addr 172.21.15.114:0/1183028636 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdf7fe700 1 --1- 172.21.15.114:0/1183028636 >> v1:172.21.15.114:6789/0 conn(0x7fdbe015bdd0 0x7fdbe00b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55354/0 (socket says 172.21.15.114:55354) 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 -- 172.21.15.114:0/1183028636 >> v1:172.21.15.172:6789/0 conn(0x7fdbe00d1c70 legacy=0x7fdbe0165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 -- 172.21.15.114:0/1183028636 >> v1:172.21.15.114:6789/0 conn(0x7fdbe015bdd0 legacy=0x7fdbe00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbde7fc700 1 -- 172.21.15.114:0/1183028636 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276977791 0 0) 0x7fdbe00b22d0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 -- 172.21.15.114:0/1183028636 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00b2890 con 0x7fdbe00b3370 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbdeffd700 1 --2- 172.21.15.114:0/1183028636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe00b3370 0x7fdbe01604e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00098f0 tx=0x7fdbd0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2312a02b2b58f4b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.001+0000 7fdbde7fc700 1 -- 172.21.15.114:0/1183028636 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000b8a0 con 0x7fdbe00b3370 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbde7fc700 1 -- 172.21.15.114:0/1183028636 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd000ba00 con 0x7fdbe00b3370 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbde7fc700 1 -- 172.21.15.114:0/1183028636 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000bd90 con 0x7fdbe00b3370 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 -- 172.21.15.114:0/1183028636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe00b3370 msgr2=0x7fdbe01604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/1183028636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe00b3370 0x7fdbe01604e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00098f0 tx=0x7fdbd0009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 -- 172.21.15.114:0/1183028636 shutdown_connections 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/1183028636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe00b3370 0x7fdbe01604e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 -- 172.21.15.114:0/1183028636 >> 172.21.15.114:0/1183028636 conn(0x7fdbe00731a0 msgr2=0x7fdbe009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 -- 172.21.15.114:0/1183028636 shutdown_connections 2023-11-24T06:25:14.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.002+0000 7fdbe5be6700 1 -- 172.21.15.114:0/1183028636 wait complete. 2023-11-24T06:25:14.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.003+0000 7fdbe5be6700 1 Processor -- start 2023-11-24T06:25:14.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.003+0000 7fdbe5be6700 1 -- start start 2023-11-24T06:25:14.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00b3370 0x7fdbe00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00d1c70 0x7fdbe00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00cab40 con 0x7fdbe015bdd0 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c8db0 con 0x7fdbe00b3370 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbe5be6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00cb490 con 0x7fdbe00d1c70 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39656/0 (socket says 172.21.15.114:39656) 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbdffff700 1 -- 172.21.15.114:0/722952213 learned_addr learned my addr 172.21.15.114:0/722952213 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.004+0000 7fdbdf7fe700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00b3370 0x7fdbe00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 -- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00d1c70 msgr2=0x7fdbe00ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00d1c70 0x7fdbe00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 -- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00b3370 msgr2=0x7fdbe00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00b3370 0x7fdbe00cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 -- 172.21.15.114:0/722952213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0009550 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbdffff700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800cc50 tx=0x7fdbd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800ad80 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800ce50 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe00cb770 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8003c40 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.005+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe00c8fc0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.006+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe00c9180 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.006+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbd80050d0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.007+0000 7fdbc7fff700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 0x7fdbcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.007+0000 7fdbdf7fe700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 0x7fdbcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.007+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbd80893a0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.007+0000 7fdbdf7fe700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 0x7fdbcc06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007fd0 tx=0x7fdbd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.011+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbd80547a0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.279+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fdbe0067f60 con 0x7fdbe015bdd0 2023-11-24T06:25:14.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.281+0000 7fdbc7fff700 1 -- 172.21.15.114:0/722952213 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdbd8057dc0 con 0x7fdbe015bdd0 2023-11-24T06:25:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 msgr2=0x7fdbcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 0x7fdbcc06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007fd0 tx=0x7fdbd4009040 comp rx=0 tx=0).stop 2023-11-24T06:25:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 msgr2=0x7fdbe015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800cc50 tx=0x7fdbd800a340 comp rx=0 tx=0).stop 2023-11-24T06:25:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 shutdown_connections 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdbcc06aed0 0x7fdbcc06d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbe015bdd0 0x7fdbe015e610 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbe00b3370 0x7fdbe00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 --2- 172.21.15.114:0/722952213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbe00d1c70 0x7fdbe00ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 >> 172.21.15.114:0/722952213 conn(0x7fdbe00731a0 msgr2=0x7fdbe00afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 shutdown_connections 2023-11-24T06:25:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.283+0000 7fdbe5be6700 1 -- 172.21.15.114:0/722952213 wait complete. 2023-11-24T06:25:14.299 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:14.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.416+0000 7f043ae0e700 1 Processor -- start 2023-11-24T06:25:14.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.416+0000 7f043ae0e700 1 -- start start 2023-11-24T06:25:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.417+0000 7f043ae0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 0x7f043406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.417+0000 7f043ae0e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04340b4350 0x7f04340af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.417+0000 7f043ae0e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340b0290 con 0x7f043406e0b0 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.417+0000 7f043ae0e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04340b4820 con 0x7f04340b4350 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.417+0000 7f043ae0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340b4b00 con 0x7f04340b59f0 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f0438baa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 0x7f043406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f04340b4350 0x7f04340af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f0438baa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 0x7f043406db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:34204/0 (socket says 172.21.15.114:34204) 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f0433fff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f043406e0b0 0x7f04340b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:48964/0 (socket says 172.21.15.114:48964) 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f0438baa700 1 -- 172.21.15.114:0/4109980737 learned_addr learned my addr 172.21.15.114:0/4109980737 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 -- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 msgr2=0x7f043406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 --2- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 0x7f043406db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 -- 172.21.15.114:0/4109980737 >> v1:172.21.15.172:6789/0 conn(0x7f043406e0b0 legacy=0x7f04340b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 -- 172.21.15.114:0/4109980737 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340ae9b0 con 0x7f04340b4350 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04337fe700 1 -- 172.21.15.114:0/4109980737 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119199608 0 0) 0x7f04340b0290 con 0x7f043406e0b0 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04393ab700 1 --2- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04340b4350 0x7f04340af190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f042c009d90 tx=0x7f042c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6860721f8648ceea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04337fe700 1 -- 172.21.15.114:0/4109980737 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00e070 con 0x7f04340b4350 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04337fe700 1 -- 172.21.15.114:0/4109980737 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00bb20 con 0x7f04340b4350 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.418+0000 7f04337fe700 1 -- 172.21.15.114:0/4109980737 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c012430 con 0x7f04340b4350 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 -- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04340b4350 msgr2=0x7f04340af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 --2- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04340b4350 0x7f04340af190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f042c009d90 tx=0x7f042c00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 -- 172.21.15.114:0/4109980737 shutdown_connections 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 --2- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04340b4350 0x7f04340af190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 --2- 172.21.15.114:0/4109980737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b59f0 0x7f043406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 -- 172.21.15.114:0/4109980737 >> 172.21.15.114:0/4109980737 conn(0x7f0434072990 msgr2=0x7f043406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 -- 172.21.15.114:0/4109980737 shutdown_connections 2023-11-24T06:25:14.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.419+0000 7f043ae0e700 1 -- 172.21.15.114:0/4109980737 wait complete. 2023-11-24T06:25:14.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.420+0000 7f043ae0e700 1 Processor -- start 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.420+0000 7f043ae0e700 1 -- start start 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 0x7f04340cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 0x7f043413e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043415a7e0 con 0x7f04340b59f0 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043413e780 con 0x7f043406e0b0 2023-11-24T06:25:14.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f043ae0e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043414ff40 con 0x7f04340b4350 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f04393ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f04393ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39670/0 (socket says 172.21.15.114:39670) 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f04393ab700 1 -- 172.21.15.114:0/2043164356 learned_addr learned my addr 172.21.15.114:0/2043164356 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f0438baa700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 0x7f04340cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f0433fff700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 0x7f043413e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.421+0000 7f04393ab700 1 -- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 msgr2=0x7f043413e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04393ab700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 0x7f043413e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04393ab700 1 -- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 msgr2=0x7f04340cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04393ab700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 0x7f04340cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04393ab700 1 -- 172.21.15.114:0/2043164356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c009a70 con 0x7f04340b59f0 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f0433fff700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 0x7f043413e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:14.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f0438baa700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 0x7f04340cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04393ab700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f042c013000 tx=0x7f042c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c012430 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c020070 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04340cfff0 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c004280 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.422+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043415ad10 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.423+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0434067f60 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.423+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f042c0043e0 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.424+0000 7f04317fa700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 0x7f041806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.424+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f042c022070 con 0x7f04340b59f0 2023-11-24T06:25:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.424+0000 7f0438baa700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 0x7f041806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.424+0000 7f0438baa700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 0x7f041806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0428009c20 tx=0x7f0428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.428+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f042c056410 con 0x7f04340b59f0 2023-11-24T06:25:14.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.621+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04340aecc0 con 0x7f04340b59f0 2023-11-24T06:25:14.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.622+0000 7f04317fa700 1 -- 172.21.15.114:0/2043164356 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12893 (secure 0 0 0) 0x7f042c059a30 con 0x7f04340b59f0 2023-11-24T06:25:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:14.625 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:11.344630+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:14.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 msgr2=0x7f041806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 0x7f041806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0428009c20 tx=0x7f0428008040 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 msgr2=0x7f043414fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f042c013000 tx=0x7f042c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 shutdown_connections 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f041806aed0 0x7f041806d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04340b59f0 0x7f043414fa00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043406e0b0 0x7f04340cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 --2- 172.21.15.114:0/2043164356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04340b4350 0x7f043413e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 >> 172.21.15.114:0/2043164356 conn(0x7f0434072990 msgr2=0x7f043406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 shutdown_connections 2023-11-24T06:25:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.624+0000 7f043ae0e700 1 -- 172.21.15.114:0/2043164356 wait complete. 2023-11-24T06:25:14.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:14.640 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.761+0000 7fbf93343700 1 Processor -- start 2023-11-24T06:25:14.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.761+0000 7fbf93343700 1 -- start start 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf93343700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf8c15e160 0x7fbf8c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf93343700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf93343700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c0b3c60 con 0x7fbf8c0afc90 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf93343700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c0b4500 con 0x7fbf8c15e160 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf93343700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0b58c0 con 0x7fbf8c0746a0 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf910df700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf8c15e160 0x7fbf8c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf910df700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf8c15e160 0x7fbf8c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39674/0 (socket says 172.21.15.114:39674) 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf910df700 1 -- 172.21.15.114:0/4237812320 learned_addr learned my addr 172.21.15.114:0/4237812320 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf908de700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf8c0afc90 0x7fbf8c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55382/0 (socket says 172.21.15.114:55382) 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf918e0700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf910df700 1 -- 172.21.15.114:0/4237812320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 msgr2=0x7fbf8c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:14.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.762+0000 7fbf910df700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf910df700 1 -- 172.21.15.114:0/4237812320 >> v1:172.21.15.114:6789/0 conn(0x7fbf8c0afc90 legacy=0x7fbf8c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf910df700 1 -- 172.21.15.114:0/4237812320 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0b0140 con 0x7fbf8c15e160 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf83fff700 1 -- 172.21.15.114:0/4237812320 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823648896 0 0) 0x7fbf8c0b3c60 con 0x7fbf8c0afc90 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf910df700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0b5da0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0098f0 tx=0x7fbf7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660da89a7d3b7d16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf83fff700 1 -- 172.21.15.114:0/4237812320 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00b8a0 con 0x7fbf8c15e160 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf83fff700 1 -- 172.21.15.114:0/4237812320 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00ba00 con 0x7fbf8c15e160 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf83fff700 1 -- 172.21.15.114:0/4237812320 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00bd90 con 0x7fbf8c15e160 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 -- 172.21.15.114:0/4237812320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c15e160 msgr2=0x7fbf8c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0b5da0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0098f0 tx=0x7fbf7c009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 -- 172.21.15.114:0/4237812320 shutdown_connections 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 --2- 172.21.15.114:0/4237812320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0b5da0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.763+0000 7fbf93343700 1 -- 172.21.15.114:0/4237812320 >> 172.21.15.114:0/4237812320 conn(0x7fbf8c0d0bf0 msgr2=0x7fbf8c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.764+0000 7fbf93343700 1 -- 172.21.15.114:0/4237812320 shutdown_connections 2023-11-24T06:25:14.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.764+0000 7fbf93343700 1 -- 172.21.15.114:0/4237812320 wait complete. 2023-11-24T06:25:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 Processor -- start 2023-11-24T06:25:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 -- start start 2023-11-24T06:25:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0c6d30 con 0x7fbf8c0afc90 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0c6f40 con 0x7fbf8c15e160 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf93343700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c0c7220 con 0x7fbf8c0746a0 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf908de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf908de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39682/0 (socket says 172.21.15.114:39682) 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.765+0000 7fbf908de700 1 -- 172.21.15.114:0/1153662640 learned_addr learned my addr 172.21.15.114:0/1153662640 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf910df700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf918e0700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 -- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 msgr2=0x7fbf8c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 -- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 msgr2=0x7fbf8c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 -- 172.21.15.114:0/1153662640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009550 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf910df700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf918e0700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf908de700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf880011e0 tx=0x7fbf8800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004c80 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88004e00 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c0c7500 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88007310 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.766+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c15b000 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.767+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c068fb0 con 0x7fbf8c0afc90 2023-11-24T06:25:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.767+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf88007470 con 0x7fbf8c0afc90 2023-11-24T06:25:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.768+0000 7fbf81ffb700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 0x7fbf7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.768+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf880890a0 con 0x7fbf8c0afc90 2023-11-24T06:25:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.768+0000 7fbf910df700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 0x7fbf7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.768+0000 7fbf910df700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 0x7fbf7006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c014040 tx=0x7fbf7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.772+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf880544a0 con 0x7fbf8c0afc90 2023-11-24T06:25:14.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.960+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf8c0c7a20 con 0x7fbf8c0afc90 2023-11-24T06:25:14.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.960+0000 7fbf81ffb700 1 -- 172.21.15.114:0/1153662640 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf8c0c7a20 con 0x7fbf8c0afc90 2023-11-24T06:25:14.963 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 msgr2=0x7fbf7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 0x7fbf7006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c014040 tx=0x7fbf7c008030 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 msgr2=0x7fbf8c1f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbf880011e0 tx=0x7fbf8800cae0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 shutdown_connections 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf7006aed0 0x7fbf7006d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf8c15e160 0x7fbf8c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf8c0746a0 0x7fbf8c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 --2- 172.21.15.114:0/1153662640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c0afc90 0x7fbf8c1f38f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 >> 172.21.15.114:0/1153662640 conn(0x7fbf8c0d0bf0 msgr2=0x7fbf8c0aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:14.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 shutdown_connections 2023-11-24T06:25:14.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:14.962+0000 7fbf93343700 1 -- 172.21.15.114:0/1153662640 wait complete. 2023-11-24T06:25:14.978 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:25:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.094+0000 7fc51110f700 1 Processor -- start 2023-11-24T06:25:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 -- start start 2023-11-24T06:25:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc50c15bdd0 0x7fc50c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c0b22d0 con 0x7fc50c0b3370 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c0b2f50 con 0x7fc50c15bdd0 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc51110f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0d2640 con 0x7fc50c0d1c70 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc50ad9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc50c15bdd0 0x7fc50c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc50a59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc50c0b3370 0x7fc50c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55398/0 (socket says 172.21.15.114:55398) 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.095+0000 7fc50a59c700 1 -- 172.21.15.114:0/3017530820 learned_addr learned my addr 172.21.15.114:0/3017530820 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50b59e700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50ad9d700 1 -- 172.21.15.114:0/3017530820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 msgr2=0x7fc50c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50ad9d700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50ad9d700 1 -- 172.21.15.114:0/3017530820 >> v1:172.21.15.114:6789/0 conn(0x7fc50c0b3370 legacy=0x7fc50c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc509d9b700 1 -- 172.21.15.114:0/3017530820 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884774512 0 0) 0x7fc50c0b22d0 con 0x7fc50c0b3370 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50ad9d700 1 -- 172.21.15.114:0/3017530820 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0b25b0 con 0x7fc50c15bdd0 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50b59e700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc50ad9d700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c1604e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc500009840 tx=0x7fc500009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b30816d6e41322a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc509d9b700 1 -- 172.21.15.114:0/3017530820 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500011830 con 0x7fc50c15bdd0 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc509d9b700 1 -- 172.21.15.114:0/3017530820 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc500011990 con 0x7fc50c15bdd0 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.096+0000 7fc509d9b700 1 -- 172.21.15.114:0/3017530820 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500011d20 con 0x7fc50c15bdd0 2023-11-24T06:25:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 -- 172.21.15.114:0/3017530820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 msgr2=0x7fc50c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c1604e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc500009840 tx=0x7fc500009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 -- 172.21.15.114:0/3017530820 shutdown_connections 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 --2- 172.21.15.114:0/3017530820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c1604e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 -- 172.21.15.114:0/3017530820 >> 172.21.15.114:0/3017530820 conn(0x7fc50c0731a0 msgr2=0x7fc50c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 -- 172.21.15.114:0/3017530820 shutdown_connections 2023-11-24T06:25:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.097+0000 7fc51110f700 1 -- 172.21.15.114:0/3017530820 wait complete. 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.098+0000 7fc51110f700 1 Processor -- start 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.098+0000 7fc51110f700 1 -- start start 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc50c0b3370 0x7fc50c0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c157780 con 0x7fc50c15bdd0 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0b6100 con 0x7fc50c0b3370 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc51110f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0b63e0 con 0x7fc50c0d1c70 2023-11-24T06:25:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39698/0 (socket says 172.21.15.114:39698) 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 -- 172.21.15.114:0/408678088 learned_addr learned my addr 172.21.15.114:0/408678088 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50ad9d700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc50c0b3370 0x7fc50c0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50a59c700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 -- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 msgr2=0x7fc50c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 -- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc50c0b3370 msgr2=0x7fc50c0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc50c0b3370 0x7fc50c0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.099+0000 7fc50b59e700 1 -- 172.21.15.114:0/408678088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000094a0 con 0x7fc50c15bdd0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc50b59e700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc50400cef0 tx=0x7fc50400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400ad00 con 0x7fc50c15bdd0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50400ae60 con 0x7fc50c15bdd0 2023-11-24T06:25:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c0b66c0 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504015680 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.100+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c0b6c80 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.101+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c067f60 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.101+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc50400d430 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.102+0000 7fc4f37fe700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 0x7fc4f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.102+0000 7fc50ad9d700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 0x7fc4f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.102+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc5040896c0 con 0x7fc50c15bdd0 2023-11-24T06:25:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.102+0000 7fc50ad9d700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 0x7fc4f806d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5000090b0 tx=0x7fc50001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.106+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc504054b50 con 0x7fc50c15bdd0 2023-11-24T06:25:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.299+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc50c0b6ea0 con 0x7fc50c15bdd0 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.299+0000 7fc4f37fe700 1 -- 172.21.15.114:0/408678088 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc504058170 con 0x7fc50c15bdd0 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:15.303 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 msgr2=0x7fc4f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 0x7fc4f806d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5000090b0 tx=0x7fc50001c040 comp rx=0 tx=0).stop 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 msgr2=0x7fc50c0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc50400cef0 tx=0x7fc50400cf20 comp rx=0 tx=0).stop 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 shutdown_connections 2023-11-24T06:25:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc4f806aed0 0x7fc4f806d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc50c15bdd0 0x7fc50c0cdc70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc50c0b3370 0x7fc50c0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 --2- 172.21.15.114:0/408678088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc50c0d1c70 0x7fc50c157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 >> 172.21.15.114:0/408678088 conn(0x7fc50c0731a0 msgr2=0x7fc50c0b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 shutdown_connections 2023-11-24T06:25:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.301+0000 7fc51110f700 1 -- 172.21.15.114:0/408678088 wait complete. 2023-11-24T06:25:15.317 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:25:15.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:25:15.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 Processor -- start 2023-11-24T06:25:15.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 -- start start 2023-11-24T06:25:15.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc8006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800b5e40 con 0x7fcc800746a0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800b05c0 con 0x7fcc800b59c0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.438+0000 7fcc861f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801470a0 con 0x7fcc800b01e0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcc800746a0 0x7fcc8006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55414/0 (socket says 172.21.15.114:55414) 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7ffff700 1 -- 172.21.15.114:0/1942949251 learned_addr learned my addr 172.21.15.114:0/1942949251 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7f7fe700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc8006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898840898 0 0) 0x7fcc800b5e40 con 0x7fcc800746a0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6c0035b0 con 0x7fcc800746a0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707632749 0 0) 0x7fcc6c0035b0 con 0x7fcc800746a0 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 msgr2=0x7fcc8006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc8006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 >> v1:172.21.15.172:6789/0 conn(0x7fcc800b59c0 legacy=0x7fcc80073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc800b4250 con 0x7fcc800746a0 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fcc780019f0 con 0x7fcc800746a0 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 >> v1:172.21.15.114:6789/0 conn(0x7fcc800746a0 legacy=0x7fcc8006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc6c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6c003fb0 0x7fcc6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc6c004d90 0x7fcc6c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801470a0 con 0x7fcc6c004d90 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c00bba0 con 0x7fcc6c003fb0 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c00bd60 con 0x7fcc800b01e0 2023-11-24T06:25:15.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7ffff700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc6c004d90 0x7fcc6c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7f7fe700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc6c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.439+0000 7fcc7effd700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6c003fb0 0x7fcc6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 -- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 msgr2=0x7fcc6c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc800b01e0 0x7fcc6c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 -- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6c003fb0 msgr2=0x7fcc6c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6c003fb0 0x7fcc6c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 -- 172.21.15.114:0/1942949251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0035b0 con 0x7fcc6c004d90 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc6c004d90 0x7fcc6c005210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a820 tx=0x7fcc6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d6708835957bc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800a2f0 con 0x7fcc6c004d90 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78012600 con 0x7fcc6c004d90 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7e7fc700 1 -- 172.21.15.114:0/1942949251 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc780129b0 con 0x7fcc6c004d90 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc861f5700 1 -- 172.21.15.114:0/1942949251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc6c004d90 msgr2=0x7fcc6c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1942949251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc6c004d90 0x7fcc6c005210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a820 tx=0x7fcc6c005880 comp rx=0 tx=0).stop 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc7ffff700 1 -- 172.21.15.114:0/1942949251 reap_dead start 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc861f5700 1 -- 172.21.15.114:0/1942949251 shutdown_connections 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc861f5700 1 -- 172.21.15.114:0/1942949251 >> 172.21.15.114:0/1942949251 conn(0x7fcc800d0b40 msgr2=0x7fcc800d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.440+0000 7fcc861f5700 1 -- 172.21.15.114:0/1942949251 shutdown_connections 2023-11-24T06:25:15.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.441+0000 7fcc861f5700 1 -- 172.21.15.114:0/1942949251 wait complete. 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.442+0000 7fcc861f5700 1 Processor -- start 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.442+0000 7fcc861f5700 1 -- start start 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc8015be80 0x7fcc80142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8015ce70 0x7fcc8015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801f47e0 con 0x7fcc8015c260 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8015d840 con 0x7fcc8015ce70 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801f3e40 con 0x7fcc8015be80 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39716/0 (socket says 172.21.15.114:39716) 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 -- 172.21.15.114:0/1764756732 learned_addr learned my addr 172.21.15.114:0/1764756732 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7f7fe700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc8015be80 0x7fcc80142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7ffff700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8015ce70 0x7fcc8015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 -- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc8015be80 msgr2=0x7fcc80142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc8015be80 0x7fcc80142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 -- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8015ce70 msgr2=0x7fcc8015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8015ce70 0x7fcc8015d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 -- 172.21.15.114:0/1764756732 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc801f4120 con 0x7fcc8015c260 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc7effd700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800f850 tx=0x7fcc6800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68017070 con 0x7fcc8015c260 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc680052e0 con 0x7fcc8015c260 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc801f4400 con 0x7fcc8015c260 2023-11-24T06:25:15.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68005490 con 0x7fcc8015c260 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.443+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8015f9b0 con 0x7fcc8015c260 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.444+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc801f4920 con 0x7fcc8015c260 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.445+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc68006860 con 0x7fcc8015c260 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.445+0000 7fcc77fff700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.445+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc68088800 con 0x7fcc8015c260 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.445+0000 7fcc7f7fe700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.445+0000 7fcc7f7fe700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc80143030 tx=0x7fcc70009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:15.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.449+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc801f4920 con 0x7fcc8015c260 2023-11-24T06:25:15.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:15.640+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc801f4920 con 0x7fcc8015c260 2023-11-24T06:25:16.295 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:16.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:16.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.461+0000 7fcc77fff700 1 -- 172.21.15.114:0/1764756732 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc801f4920 con 0x7fcc8015c260 2023-11-24T06:25:16.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.465+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 msgr2=0x7fcc6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:16.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.465+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc80143030 tx=0x7fcc70009210 comp rx=0 tx=0).stop 2023-11-24T06:25:16.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 msgr2=0x7fcc8015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:16.469 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800f850 tx=0x7fcc6800fb60 comp rx=0 tx=0).stop 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 shutdown_connections 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc6006aed0 0x7fcc6006d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8015ce70 0x7fcc8015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc8015be80 0x7fcc80142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 --2- 172.21.15.114:0/1764756732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc8015c260 0x7fcc8015f160 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 >> 172.21.15.114:0/1764756732 conn(0x7fcc800d0b40 msgr2=0x7fcc80071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 shutdown_connections 2023-11-24T06:25:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.466+0000 7fcc861f5700 1 -- 172.21.15.114:0/1764756732 wait complete. 2023-11-24T06:25:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2023-11-24T06:25:16.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:25:16.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.615+0000 7fb85ca78700 1 Processor -- start 2023-11-24T06:25:16.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.615+0000 7fb85ca78700 1 -- start start 2023-11-24T06:25:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb85815e160 0x7fb8580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb8580afc90 0x7fb858073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8580744d0 0x7fb85806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580b4500 con 0x7fb8580afc90 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb858074960 con 0x7fb8580744d0 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.616+0000 7fb85ca78700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b3c60 con 0x7fb85815e160 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb8580afc90 0x7fb858073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb8580afc90 0x7fb858073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39722/0 (socket says 172.21.15.114:39722) 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 -- 172.21.15.114:0/1895856654 learned_addr learned my addr 172.21.15.114:0/1895856654 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 -- 172.21.15.114:0/1895856654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb85815e160 msgr2=0x7fb8580b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 --2- 172.21.15.114:0/1895856654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb85815e160 0x7fb8580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 -- 172.21.15.114:0/1895856654 >> v2:172.21.15.172:3300/0 conn(0x7fb8580744d0 msgr2=0x7fb85806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb856d9d700 1 --2- 172.21.15.114:0/1895856654 >> v2:172.21.15.172:3300/0 conn(0x7fb8580744d0 0x7fb85806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 --2- 172.21.15.114:0/1895856654 >> v2:172.21.15.172:3300/0 conn(0x7fb8580744d0 0x7fb85806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 -- 172.21.15.114:0/1895856654 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580b58c0 con 0x7fb8580afc90 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb855d9b700 1 --2- 172.21.15.114:0/1895856654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580afc90 0x7fb858073f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d1cd0 tx=0x7fb848009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99554086f2ac24d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb85559a700 1 -- 172.21.15.114:0/1895856654 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800b870 con 0x7fb8580afc90 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb85559a700 1 -- 172.21.15.114:0/1895856654 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84800b9d0 con 0x7fb8580afc90 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.617+0000 7fb85559a700 1 -- 172.21.15.114:0/1895856654 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800bd60 con 0x7fb8580afc90 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 -- 172.21.15.114:0/1895856654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580afc90 msgr2=0x7fb858073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 --2- 172.21.15.114:0/1895856654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580afc90 0x7fb858073f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb8580d1cd0 tx=0x7fb848009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 -- 172.21.15.114:0/1895856654 shutdown_connections 2023-11-24T06:25:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 --2- 172.21.15.114:0/1895856654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb85815e160 0x7fb8580b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 --2- 172.21.15.114:0/1895856654 >> v2:172.21.15.172:3300/0 conn(0x7fb8580744d0 0x7fb85806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 --2- 172.21.15.114:0/1895856654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580afc90 0x7fb858073f90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 -- 172.21.15.114:0/1895856654 >> 172.21.15.114:0/1895856654 conn(0x7fb8580d0bf0 msgr2=0x7fb8580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 -- 172.21.15.114:0/1895856654 shutdown_connections 2023-11-24T06:25:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.618+0000 7fb85ca78700 1 -- 172.21.15.114:0/1895856654 wait complete. 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.619+0000 7fb85ca78700 1 Processor -- start 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.619+0000 7fb85ca78700 1 -- start start 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 0x7fb8581485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 0x7fb8581603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581528e0 con 0x7fb8580744d0 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85815e730 con 0x7fb85815e160 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85ca78700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb858160f10 con 0x7fb8580afc90 2023-11-24T06:25:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39730/0 (socket says 172.21.15.114:39730) 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb85659c700 1 -- 172.21.15.114:0/2902728088 learned_addr learned my addr 172.21.15.114:0/2902728088 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb855d9b700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 0x7fb8581485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.620+0000 7fb856d9d700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 0x7fb8581603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 -- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 msgr2=0x7fb8581485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 0x7fb8581485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 -- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 msgr2=0x7fb8581603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 0x7fb8581603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 -- 172.21.15.114:0/2902728088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb848009550 con 0x7fb8580744d0 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb855d9b700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 0x7fb8581485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb856d9d700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 0x7fb8581603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85659c700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c011830 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c011990 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85815e9d0 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c011d40 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.621+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85815ef90 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.622+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb858068fb0 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.622+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb84c01a400 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.623+0000 7fb846ffd700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 0x7fb83806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.623+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb84c089460 con 0x7fb8580744d0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.623+0000 7fb855d9b700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 0x7fb83806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.623+0000 7fb855d9b700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 0x7fb83806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb858160370 tx=0x7fb848008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:16.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.627+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb84c054700 con 0x7fb8580744d0 2023-11-24T06:25:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:16.818+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb858068290 con 0x7fb8580744d0 2023-11-24T06:25:17.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-24T06:25:17.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-24T06:25:17.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-24T06:25:17.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-24T06:25:17.467 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.463+0000 7fb846ffd700 1 -- 172.21.15.114:0/2902728088 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb84c057d20 con 0x7fb8580744d0 2023-11-24T06:25:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.466+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 msgr2=0x7fb83806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.466+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 0x7fb83806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb858160370 tx=0x7fb848008030 comp rx=0 tx=0).stop 2023-11-24T06:25:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.466+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 msgr2=0x7fb85815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.466+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b4a0 tx=0x7fb84c00b860 comp rx=0 tx=0).stop 2023-11-24T06:25:17.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 shutdown_connections 2023-11-24T06:25:17.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb83806aed0 0x7fb83806d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb85815e160 0x7fb8581603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.471 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8580744d0 0x7fb85815f4b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 --2- 172.21.15.114:0/2902728088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8580afc90 0x7fb8581485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.467+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 >> 172.21.15.114:0/2902728088 conn(0x7fb8580d0bf0 msgr2=0x7fb85806e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.468+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 shutdown_connections 2023-11-24T06:25:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.468+0000 7fb85ca78700 1 -- 172.21.15.114:0/2902728088 wait complete. 2023-11-24T06:25:17.478 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2023-11-24T06:25:17.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:25:17.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 Processor -- start 2023-11-24T06:25:17.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 -- start start 2023-11-24T06:25:17.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd42015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200ae5e0 con 0x7fd4201587f0 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42015be30 con 0x7fd4200b3c60 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd425546700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015c110 con 0x7fd42015b370 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd41effd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd4201587f0 0x7fd42006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55444/0 (socket says 172.21.15.114:55444) 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.623+0000 7fd41effd700 1 -- 172.21.15.114:0/1935711341 learned_addr learned my addr 172.21.15.114:0/1935711341 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839387282 0 0) 0x7fd4200ae5e0 con 0x7fd4201587f0 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4080035b0 con 0x7fd4201587f0 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41e7fc700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd42015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190751829 0 0) 0x7fd4080035b0 con 0x7fd4201587f0 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 msgr2=0x7fd42015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:17.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd42015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 >> v1:172.21.15.172:6789/0 conn(0x7fd4200b3c60 legacy=0x7fd4200af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200ae860 con 0x7fd4201587f0 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fd414001670 con 0x7fd4201587f0 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 >> v1:172.21.15.114:6789/0 conn(0x7fd4201587f0 legacy=0x7fd42006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd408003a30 0x7fd408003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408004850 0x7fd408004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd408005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015c110 con 0x7fd408003a30 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40800bac0 con 0x7fd408004850 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40800bc80 con 0x7fd42015b370 2023-11-24T06:25:17.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd408003a30 0x7fd408003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41f7fe700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd408005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41e7fc700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408004850 0x7fd408004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 -- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 msgr2=0x7fd408005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd42015b370 0x7fd408005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 -- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408004850 msgr2=0x7fd408004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408004850 0x7fd408004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.624+0000 7fd41effd700 1 -- 172.21.15.114:0/1935711341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080035b0 con 0x7fd408003a30 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd41effd700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd408003a30 0x7fd408003e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c210 tx=0x7fd408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d755ec2736d485e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414007c10 con 0x7fd408003a30 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41400cdb0 con 0x7fd408003a30 2023-11-24T06:25:17.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd41dffb700 1 -- 172.21.15.114:0/1935711341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414005ba0 con 0x7fd408003a30 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 -- 172.21.15.114:0/1935711341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd408003a30 msgr2=0x7fd408003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 --2- 172.21.15.114:0/1935711341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd408003a30 0x7fd408003e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c210 tx=0x7fd408005880 comp rx=0 tx=0).stop 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd41f7fe700 1 -- 172.21.15.114:0/1935711341 reap_dead start 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 -- 172.21.15.114:0/1935711341 shutdown_connections 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 -- 172.21.15.114:0/1935711341 >> 172.21.15.114:0/1935711341 conn(0x7fd4200d0bf0 msgr2=0x7fd4200d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 -- 172.21.15.114:0/1935711341 shutdown_connections 2023-11-24T06:25:17.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.625+0000 7fd425546700 1 -- 172.21.15.114:0/1935711341 wait complete. 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.627+0000 7fd425546700 1 Processor -- start 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.627+0000 7fd425546700 1 -- start start 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408003a30 0x7fd42009ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4201456e0 0x7fd42009d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42009dff0 con 0x7fd42009db70 2023-11-24T06:25:17.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420146010 con 0x7fd408003a30 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd425546700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4201462f0 con 0x7fd4201456e0 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39752/0 (socket says 172.21.15.114:39752) 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 -- 172.21.15.114:0/341031144 learned_addr learned my addr 172.21.15.114:0/341031144 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 -- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4201456e0 msgr2=0x7fd42009d630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4201456e0 0x7fd42009d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 -- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408003a30 msgr2=0x7fd42009ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408003a30 0x7fd42009ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.628+0000 7fd41f7fe700 1 -- 172.21.15.114:0/341031144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4201465d0 con 0x7fd42009db70 2023-11-24T06:25:17.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd41f7fe700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd418000f80 tx=0x7fd41800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418003c10 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd418003d90 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4201468b0 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418005310 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.629+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420145b60 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.630+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd420159530 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.630+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd418005470 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.631+0000 7fd4077fe700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 0x7fd40c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.631+0000 7fd41effd700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 0x7fd40c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.631+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd4180896b0 con 0x7fd42009db70 2023-11-24T06:25:17.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.631+0000 7fd41effd700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 0x7fd40c06d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c210 tx=0x7fd414007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:17.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.635+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd418054730 con 0x7fd42009db70 2023-11-24T06:25:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:17.867+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd420068fb0 con 0x7fd42009db70 2023-11-24T06:25:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.490+0000 7fd4077fe700 1 -- 172.21.15.114:0/341031144 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd418057d50 con 0x7fd42009db70 2023-11-24T06:25:18.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.497+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 msgr2=0x7fd40c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.497+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 0x7fd40c06d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c210 tx=0x7fd414007000 comp rx=0 tx=0).stop 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.497+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 msgr2=0x7fd4200b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.497+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd418000f80 tx=0x7fd41800ee80 comp rx=0 tx=0).stop 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 shutdown_connections 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd40c06af00 0x7fd40c06d3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd42009db70 0x7fd4200b0ea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd408003a30 0x7fd42009ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.501 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 --2- 172.21.15.114:0/341031144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4201456e0 0x7fd42009d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 >> 172.21.15.114:0/341031144 conn(0x7fd4200d0bf0 msgr2=0x7fd42015a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:18.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.498+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 shutdown_connections 2023-11-24T06:25:18.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.500+0000 7fd425546700 1 -- 172.21.15.114:0/341031144 wait complete. 2023-11-24T06:25:18.511 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:25:18.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:18.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.648+0000 7f5e6b4bf700 1 Processor -- start 2023-11-24T06:25:18.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.648+0000 7f5e6b4bf700 1 -- start start 2023-11-24T06:25:18.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6b4bf700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e6406c710 0x7f5e640b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6b4bf700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640ae430 con 0x7f5e6415e080 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6b4bf700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640ae710 con 0x7f5e640af4d0 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6b4bf700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e640af0b0 con 0x7f5e6406c710 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6925b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5e6415e080 0x7f5e6406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55478/0 (socket says 172.21.15.114:55478) 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e6925b700 1 -- 172.21.15.114:0/213944012 learned_addr learned my addr 172.21.15.114:0/213944012 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830484184 0 0) 0x7f5e640ae430 con 0x7f5e6415e080 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e500035b0 con 0x7f5e6415e080 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e68a5a700 1 --2- 172.21.15.114:0/213944012 >> v2:172.21.15.172:3300/0 conn(0x7f5e6406c710 0x7f5e640b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 848076470 0 0) 0x7f5e500035b0 con 0x7f5e6415e080 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 >> v2:172.21.15.172:3300/0 conn(0x7f5e6406c710 msgr2=0x7f5e640b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 --2- 172.21.15.114:0/213944012 >> v2:172.21.15.172:3300/0 conn(0x7f5e6406c710 0x7f5e640b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 >> v1:172.21.15.172:6789/0 conn(0x7f5e640af4d0 legacy=0x7f5e6406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640ae9f0 con 0x7f5e6415e080 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f5e60001670 con 0x7f5e6415e080 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 >> v1:172.21.15.114:6789/0 conn(0x7f5e6415e080 legacy=0x7f5e6406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50003a00 0x7f5e50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50004850 0x7f5e50004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 0x7f5e5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640af0b0 con 0x7f5e50005210 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5000c0b0 con 0x7f5e50003a00 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.649+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5000c270 con 0x7f5e50004850 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 0x7f5e5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e6925b700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50003a00 0x7f5e50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e68a5a700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50004850 0x7f5e50004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 -- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50004850 msgr2=0x7f5e50004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50004850 0x7f5e50004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 -- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50003a00 msgr2=0x7f5e50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50003a00 0x7f5e50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 -- 172.21.15.114:0/213944012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500035b0 con 0x7f5e50005210 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 -- 172.21.15.114:0/213944012 reap_dead start 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 0x7f5e5000ba10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c190 tx=0x7f5e50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a3f4677d78527d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c001db0 con 0x7f5e50005210 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c00ccf0 con 0x7f5e50005210 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e5bfff700 1 -- 172.21.15.114:0/213944012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c003c40 con 0x7f5e50005210 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 msgr2=0x7f5e5000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.650+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 0x7f5e5000ba10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c190 tx=0x7f5e50005880 comp rx=0 tx=0).stop 2023-11-24T06:25:18.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.651+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/213944012 shutdown_connections 2023-11-24T06:25:18.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.651+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/213944012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e50005210 0x7f5e5000ba10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.651+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/213944012 >> 172.21.15.114:0/213944012 conn(0x7f5e64071b90 msgr2=0x7f5e64071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:18.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.651+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/213944012 shutdown_connections 2023-11-24T06:25:18.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.651+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/213944012 wait complete. 2023-11-24T06:25:18.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.652+0000 7f5e6b4bf700 1 Processor -- start 2023-11-24T06:25:18.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.652+0000 7f5e6b4bf700 1 -- start start 2023-11-24T06:25:18.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50005210 0x7f5e641f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64160cb0 0x7f5e64158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6415e7d0 con 0x7f5e6414ec80 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641f7630 con 0x7f5e64160cb0 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641f7910 con 0x7f5e50005210 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39786/0 (socket says 172.21.15.114:39786) 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6925b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50005210 0x7f5e641f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 -- 172.21.15.114:0/1208828925 learned_addr learned my addr 172.21.15.114:0/1208828925 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e68a5a700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64160cb0 0x7f5e64158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 -- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50005210 msgr2=0x7f5e641f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50005210 0x7f5e641f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 -- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64160cb0 msgr2=0x7f5e64158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64160cb0 0x7f5e64158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 -- 172.21.15.114:0/1208828925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e641f7bf0 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e69a5c700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005040 tx=0x7f5e5c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0052d0 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c005430 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6415e460 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.653+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0057a0 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.654+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e64158760 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.655+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6415e620 con 0x7f5e6414ec80 2023-11-24T06:25:18.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.655+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e5c01d480 con 0x7f5e6414ec80 2023-11-24T06:25:18.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.655+0000 7f5e59ffb700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 0x7f5e4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.655+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e5c08a050 con 0x7f5e6414ec80 2023-11-24T06:25:18.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.655+0000 7f5e6925b700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 0x7f5e4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.656+0000 7f5e6925b700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 0x7f5e4806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e60000c00 tx=0x7f5e60007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:18.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.660+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e5c055240 con 0x7f5e6414ec80 2023-11-24T06:25:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.853+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e640753f0 con 0x7f5e6414ec80 2023-11-24T06:25:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.854+0000 7f5e59ffb700 1 -- 172.21.15.114:0/1208828925 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f5e5c058860 con 0x7f5e6414ec80 2023-11-24T06:25:18.858 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:18.858 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":33,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:17.869654+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:18.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 msgr2=0x7f5e4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 0x7f5e4806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e60000c00 tx=0x7f5e60007980 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 msgr2=0x7f5e6406e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005040 tx=0x7f5e5c0014a0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 shutdown_connections 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5e4806aed0 0x7f5e4806d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e6414ec80 0x7f5e6406e610 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e50005210 0x7f5e641f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.856+0000 7f5e6b4bf700 1 --2- 172.21.15.114:0/1208828925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64160cb0 0x7f5e64158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.857+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 >> 172.21.15.114:0/1208828925 conn(0x7f5e64071b90 msgr2=0x7f5e640b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:18.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.857+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 shutdown_connections 2023-11-24T06:25:18.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.857+0000 7f5e6b4bf700 1 -- 172.21.15.114:0/1208828925 wait complete. 2023-11-24T06:25:18.872 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.989+0000 7f3a30f30700 1 Processor -- start 2023-11-24T06:25:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.989+0000 7f3a30f30700 1 -- start start 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a30f30700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2c15c160 0x7f3a2c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a30f30700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a2c0b2870 0x7f3a2c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a30f30700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c15c540 con 0x7f3a2c0b02e0 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a30f30700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b0750 con 0x7f3a2c15c160 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a30f30700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b2ee0 con 0x7f3a2c0b2870 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2c15c160 0x7f3a2c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2ad9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3a2c0b02e0 0x7f3a2c0ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55502/0 (socket says 172.21.15.114:55502) 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3a2c15c160 0x7f3a2c0622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39790/0 (socket says 172.21.15.114:39790) 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2ad9d700 1 -- 172.21.15.114:0/2896802852 learned_addr learned my addr 172.21.15.114:0/2896802852 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 -- 172.21.15.114:0/2896802852 >> v2:172.21.15.172:3300/0 conn(0x7f3a2c0b2870 msgr2=0x7f3a2c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 --2- 172.21.15.114:0/2896802852 >> v2:172.21.15.172:3300/0 conn(0x7f3a2c0b2870 0x7f3a2c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 -- 172.21.15.114:0/2896802852 >> v1:172.21.15.114:6789/0 conn(0x7f3a2c0b02e0 legacy=0x7f3a2c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 -- 172.21.15.114:0/2896802852 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0add40 con 0x7f3a2c15c160 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2959a700 1 -- 172.21.15.114:0/2896802852 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755113 0 0) 0x7f3a2c15c540 con 0x7f3a2c0b02e0 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2a59c700 1 --2- 172.21.15.114:0/2896802852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c0622a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0b19a0 tx=0x7f3a1c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d13bb818418f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2959a700 1 -- 172.21.15.114:0/2896802852 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011830 con 0x7f3a2c15c160 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2959a700 1 -- 172.21.15.114:0/2896802852 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c011990 con 0x7f3a2c15c160 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.990+0000 7f3a2959a700 1 -- 172.21.15.114:0/2896802852 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c011d20 con 0x7f3a2c15c160 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 -- 172.21.15.114:0/2896802852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 msgr2=0x7f3a2c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2896802852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c0622a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0b19a0 tx=0x7f3a1c009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 -- 172.21.15.114:0/2896802852 shutdown_connections 2023-11-24T06:25:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2896802852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c0622a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2896802852 >> v2:172.21.15.172:3300/0 conn(0x7f3a2c0b2870 0x7f3a2c0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 -- 172.21.15.114:0/2896802852 >> 172.21.15.114:0/2896802852 conn(0x7f3a2c0b57b0 msgr2=0x7f3a2c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 -- 172.21.15.114:0/2896802852 shutdown_connections 2023-11-24T06:25:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.991+0000 7f3a30f30700 1 -- 172.21.15.114:0/2896802852 wait complete. 2023-11-24T06:25:18.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.993+0000 7f3a30f30700 1 Processor -- start 2023-11-24T06:25:18.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.993+0000 7f3a30f30700 1 -- start start 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c0b02e0 0x7f3a2c1f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0b2870 0x7f3a2c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd790 con 0x7f3a2c15c160 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd1d0 con 0x7f3a2c0b02e0 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd4b0 con 0x7f3a2c0b2870 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39800/0 (socket says 172.21.15.114:39800) 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 -- 172.21.15.114:0/2272815604 learned_addr learned my addr 172.21.15.114:0/2272815604 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2a59c700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c0b02e0 0x7f3a2c1f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 -- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0b2870 msgr2=0x7f3a2c0cc710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0b2870 0x7f3a2c0cc710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 -- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c0b02e0 msgr2=0x7f3a2c1f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c0b02e0 0x7f3a2c1f0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 -- 172.21.15.114:0/2272815604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0094a0 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a2ad9d700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3a24002230 tx=0x7f3a2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ac20 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2400ada0 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24007310 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c15d510 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.994+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c1f12e0 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.995+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c1f14d0 con 0x7f3a2c15c160 2023-11-24T06:25:19.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.996+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a24007470 con 0x7f3a2c15c160 2023-11-24T06:25:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.996+0000 7f3a1affd700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 0x7f3a0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.996+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a24089100 con 0x7f3a2c15c160 2023-11-24T06:25:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.996+0000 7f3a2a59c700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 0x7f3a0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:18.997+0000 7f3a2a59c700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 0x7f3a0c06d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b60 tx=0x7f3a1c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:19.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.001+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a240541c0 con 0x7f3a2c15c160 2023-11-24T06:25:19.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.191+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a2c067f60 con 0x7f3a2c15c160 2023-11-24T06:25:19.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.193+0000 7f3a1affd700 1 -- 172.21.15.114:0/2272815604 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f3a240577e0 con 0x7f3a2c15c160 2023-11-24T06:25:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:19.197 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":33,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:17.869654+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 msgr2=0x7f3a0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 0x7f3a0c06d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b60 tx=0x7f3a1c008030 comp rx=0 tx=0).stop 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 msgr2=0x7f3a2c154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3a24002230 tx=0x7f3a2400a060 comp rx=0 tx=0).stop 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 shutdown_connections 2023-11-24T06:25:19.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3a0c06aed0 0x7f3a0c06d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a2c15c160 0x7f3a2c154610 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c0b02e0 0x7f3a2c1f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 --2- 172.21.15.114:0/2272815604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0b2870 0x7f3a2c0cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 >> 172.21.15.114:0/2272815604 conn(0x7f3a2c0b57b0 msgr2=0x7f3a2c06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 shutdown_connections 2023-11-24T06:25:19.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.195+0000 7f3a30f30700 1 -- 172.21.15.114:0/2272815604 wait complete. 2023-11-24T06:25:19.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:25:19.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.330+0000 7f45cb4a0700 1 Processor -- start 2023-11-24T06:25:19.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 -- start start 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f45c40b4310 0x7f45c40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45c40ce430 0x7f45c40ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40d1200 con 0x7f45c40b4310 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40cee20 con 0x7f45c40ce430 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45cb4a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415e460 con 0x7f45c415e080 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.331+0000 7f45c8a3b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f45c40b4310 0x7f45c40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f45c40b4310 0x7f45c40d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39804/0 (socket says 172.21.15.114:39804) 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 -- 172.21.15.114:0/3167570884 learned_addr learned my addr 172.21.15.114:0/3167570884 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c9a3d700 1 --2- 172.21.15.114:0/3167570884 >> v2:172.21.15.172:3300/0 conn(0x7f45c40ce430 0x7f45c40ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c923c700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 -- 172.21.15.114:0/3167570884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 msgr2=0x7f45c406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 -- 172.21.15.114:0/3167570884 >> v2:172.21.15.172:3300/0 conn(0x7f45c40ce430 msgr2=0x7f45c40ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/3167570884 >> v2:172.21.15.172:3300/0 conn(0x7f45c40ce430 0x7f45c40ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 -- 172.21.15.114:0/3167570884 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c406ba50 con 0x7f45c40b4310 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c923c700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c40d0900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45c40725a0 tx=0x7f45c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcf501987ee4e0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45bbfff700 1 -- 172.21.15.114:0/3167570884 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0011830 con 0x7f45c40b4310 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45bbfff700 1 -- 172.21.15.114:0/3167570884 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0011990 con 0x7f45c40b4310 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.332+0000 7f45bbfff700 1 -- 172.21.15.114:0/3167570884 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0011d20 con 0x7f45c40b4310 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 -- 172.21.15.114:0/3167570884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 msgr2=0x7f45c40d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c40d0900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45c40725a0 tx=0x7f45c0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 -- 172.21.15.114:0/3167570884 shutdown_connections 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/3167570884 >> v2:172.21.15.172:3300/0 conn(0x7f45c40ce430 0x7f45c40ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/3167570884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c40d0900 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 -- 172.21.15.114:0/3167570884 >> 172.21.15.114:0/3167570884 conn(0x7f45c40b3780 msgr2=0x7f45c40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 -- 172.21.15.114:0/3167570884 shutdown_connections 2023-11-24T06:25:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.333+0000 7f45cb4a0700 1 -- 172.21.15.114:0/3167570884 wait complete. 2023-11-24T06:25:19.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.334+0000 7f45cb4a0700 1 Processor -- start 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.334+0000 7f45cb4a0700 1 -- start start 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 0x7f45c4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c4157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4158b00 con 0x7f45c40b4310 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415f3f0 con 0x7f45c40ce430 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415aff0 con 0x7f45c415e080 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39810/0 (socket says 172.21.15.114:39810) 2023-11-24T06:25:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 -- 172.21.15.114:0/2343310463 learned_addr learned my addr 172.21.15.114:0/2343310463 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c9a3d700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 0x7f45c4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 -- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 msgr2=0x7f45c4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c4157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 -- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 msgr2=0x7f45c4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 0x7f45c4157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 -- 172.21.15.114:0/2343310463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009550 con 0x7f45c40b4310 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c9a3d700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 0x7f45c4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:19.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45c923c700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e810 tx=0x7f45b400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400c020 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4004250 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c415f640 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.335+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40036a0 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.336+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c415fc00 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.336+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4068fb0 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.337+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45b4003800 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.337+0000 7f45b9ffb700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 0x7f45a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.337+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45b4089380 con 0x7f45c40b4310 2023-11-24T06:25:19.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.337+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 0x7f45a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:19.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.338+0000 7f45c8a3b700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 0x7f45a806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45c40b4750 tx=0x7f45c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:19.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.342+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45b4054430 con 0x7f45c40b4310 2023-11-24T06:25:19.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:19.579+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f45c40b7ab0 con 0x7f45c40b4310 2023-11-24T06:25:20.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.501+0000 7f45b9ffb700 1 -- 172.21.15.114:0/2343310463 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f45b4057a50 con 0x7f45c40b4310 2023-11-24T06:25:20.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 msgr2=0x7f45a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 0x7f45a806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f45c40b4750 tx=0x7f45c0008030 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 msgr2=0x7f45c4154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e810 tx=0x7f45b400ebd0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 shutdown_connections 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f45a806af00 0x7f45a806d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f45c415e080 0x7f45c4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45c40b4310 0x7f45c4154000 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 --2- 172.21.15.114:0/2343310463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45c40ce430 0x7f45c4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 >> 172.21.15.114:0/2343310463 conn(0x7f45c40b3780 msgr2=0x7f45c40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 shutdown_connections 2023-11-24T06:25:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.504+0000 7f45cb4a0700 1 -- 172.21.15.114:0/2343310463 wait complete. 2023-11-24T06:25:20.522 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:25:20.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:25:20.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.647+0000 7f6f4ec7b700 1 Processor -- start 2023-11-24T06:25:20.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.647+0000 7f6f4ec7b700 1 -- start start 2023-11-24T06:25:20.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ec7b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6f4815e160 0x7f6f480b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ec7b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480afc90 0x7f6f48073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ec7b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480b3c60 con 0x7f6f480744d0 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ec7b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f480b4500 con 0x7f6f4815e160 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ec7b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48074960 con 0x7f6f480afc90 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ca17700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6f4815e160 0x7f6f480b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4d218700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6f480744d0 0x7f6f4806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55526/0 (socket says 172.21.15.114:55526) 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4ca17700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6f4815e160 0x7f6f480b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39818/0 (socket says 172.21.15.114:39818) 2023-11-24T06:25:20.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.648+0000 7f6f4d218700 1 -- 172.21.15.114:0/69655921 learned_addr learned my addr 172.21.15.114:0/69655921 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f47fff700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480afc90 0x7f6f48073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f4ca17700 1 -- 172.21.15.114:0/69655921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480afc90 msgr2=0x7f6f48073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f4ca17700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480afc90 0x7f6f48073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f4ca17700 1 -- 172.21.15.114:0/69655921 >> v1:172.21.15.114:6789/0 conn(0x7f6f480744d0 legacy=0x7f6f4806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f477fe700 1 -- 172.21.15.114:0/69655921 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114053428 0 0) 0x7f6f480b3c60 con 0x7f6f480744d0 2023-11-24T06:25:20.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f4ca17700 1 -- 172.21.15.114:0/69655921 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480b58c0 con 0x7f6f4815e160 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f4ca17700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f480b5da0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009840 tx=0x7f6f3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199925f7c4b33d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f477fe700 1 -- 172.21.15.114:0/69655921 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011830 con 0x7f6f4815e160 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f477fe700 1 -- 172.21.15.114:0/69655921 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c011990 con 0x7f6f4815e160 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.649+0000 7f6f477fe700 1 -- 172.21.15.114:0/69655921 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011d20 con 0x7f6f4815e160 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/69655921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 msgr2=0x7f6f480b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f480b5da0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009840 tx=0x7f6f3c009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/69655921 shutdown_connections 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f480b5da0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/69655921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480afc90 0x7f6f48073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/69655921 >> 172.21.15.114:0/69655921 conn(0x7f6f480d0bf0 msgr2=0x7f6f480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/69655921 shutdown_connections 2023-11-24T06:25:20.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.650+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/69655921 wait complete. 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.651+0000 7f6f4ec7b700 1 Processor -- start 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.651+0000 7f6f4ec7b700 1 -- start start 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480744d0 0x7f6f48161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 0x7f6f480c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48160f10 con 0x7f6f4815e160 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c6710 con 0x7f6f480afc90 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f48160c90 con 0x7f6f480744d0 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:20.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39826/0 (socket says 172.21.15.114:39826) 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 -- 172.21.15.114:0/206941948 learned_addr learned my addr 172.21.15.114:0/206941948 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f47fff700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 0x7f6f480c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 -- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480744d0 msgr2=0x7f6f48161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480744d0 0x7f6f48161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 -- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 msgr2=0x7f6f480c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 0x7f6f480c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 -- 172.21.15.114:0/206941948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0094a0 con 0x7f6f4815e160 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f47fff700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 0x7f6f480c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4d218700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000cc70 tx=0x7f6f4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000ada0 con 0x7f6f4815e160 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4000ce70 con 0x7f6f4815e160 2023-11-24T06:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480c5d70 con 0x7f6f4815e160 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400072f0 con 0x7f6f4815e160 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.652+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f480c6330 con 0x7f6f4815e160 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.654+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f480b51c0 con 0x7f6f4815e160 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.654+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f40007450 con 0x7f6f4815e160 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.654+0000 7f6f457fa700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 0x7f6f2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:20.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.654+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f400890c0 con 0x7f6f4815e160 2023-11-24T06:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.654+0000 7f6f4ca17700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 0x7f6f2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.655+0000 7f6f4ca17700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 0x7f6f2c06d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000c00 tx=0x7f6f3c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.659+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f40054180 con 0x7f6f4815e160 2023-11-24T06:25:20.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:20.894+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6f48068fb0 con 0x7f6f4815e160 2023-11-24T06:25:21.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.501+0000 7f6f457fa700 1 -- 172.21.15.114:0/206941948 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f400577a0 con 0x7f6f4815e160 2023-11-24T06:25:21.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 msgr2=0x7f6f2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 0x7f6f2c06d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000c00 tx=0x7f6f3c010000 comp rx=0 tx=0).stop 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 msgr2=0x7f6f48160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000cc70 tx=0x7f6f4000a340 comp rx=0 tx=0).stop 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 shutdown_connections 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f2c06aed0 0x7f6f2c06d390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f4815e160 0x7f6f48160750 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f480744d0 0x7f6f48161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 --2- 172.21.15.114:0/206941948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f480afc90 0x7f6f480c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 >> 172.21.15.114:0/206941948 conn(0x7f6f480d0bf0 msgr2=0x7f6f4815d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:21.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 shutdown_connections 2023-11-24T06:25:21.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.504+0000 7f6f4ec7b700 1 -- 172.21.15.114:0/206941948 wait complete. 2023-11-24T06:25:21.521 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:25:21.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:21.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.641+0000 7f66144c6700 1 Processor -- start 2023-11-24T06:25:21.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.641+0000 7f66144c6700 1 -- start start 2023-11-24T06:25:21.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f660c15e080 0x7f660c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f660c06c000 0x7f660c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c0af0b0 con 0x7f660c15e080 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c06c9f0 con 0x7f660c06c000 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f66144c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0b05a0 con 0x7f660c0af4d0 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f6612262700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f660c15e080 0x7f660c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f6612262700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f660c15e080 0x7f660c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39830/0 (socket says 172.21.15.114:39830) 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f6612262700 1 -- 172.21.15.114:0/3128863348 learned_addr learned my addr 172.21.15.114:0/3128863348 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f6612a63700 1 --2- 172.21.15.114:0/3128863348 >> v2:172.21.15.172:3300/0 conn(0x7f660c06c000 0x7f660c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.642+0000 7f6611a61700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 -- 172.21.15.114:0/3128863348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 msgr2=0x7f660c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 -- 172.21.15.114:0/3128863348 >> v2:172.21.15.172:3300/0 conn(0x7f660c06c000 msgr2=0x7f660c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 --2- 172.21.15.114:0/3128863348 >> v2:172.21.15.172:3300/0 conn(0x7f660c06c000 0x7f660c06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.648 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 -- 172.21.15.114:0/3128863348 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0ae430 con 0x7f660c15e080 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6612262700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c0b5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f660c09eea0 tx=0x7f65fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ada4cc6f86463b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6611260700 1 -- 172.21.15.114:0/3128863348 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00b860 con 0x7f660c15e080 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6611260700 1 -- 172.21.15.114:0/3128863348 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00b9c0 con 0x7f660c15e080 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f6611260700 1 -- 172.21.15.114:0/3128863348 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00bd50 con 0x7f660c15e080 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 -- 172.21.15.114:0/3128863348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 msgr2=0x7f660c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c0b5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f660c09eea0 tx=0x7f65fc009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 -- 172.21.15.114:0/3128863348 shutdown_connections 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 --2- 172.21.15.114:0/3128863348 >> v2:172.21.15.172:3300/0 conn(0x7f660c06c000 0x7f660c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c0b5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 --2- 172.21.15.114:0/3128863348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.643+0000 7f66144c6700 1 -- 172.21.15.114:0/3128863348 >> 172.21.15.114:0/3128863348 conn(0x7f660c071b90 msgr2=0x7f660c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:21.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.644+0000 7f66144c6700 1 -- 172.21.15.114:0/3128863348 shutdown_connections 2023-11-24T06:25:21.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.644+0000 7f66144c6700 1 -- 172.21.15.114:0/3128863348 wait complete. 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 Processor -- start 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 -- start start 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f660c06c000 0x7f660c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c153190 con 0x7f660c15e080 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c15f7c0 con 0x7f660c06c000 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.645+0000 7f66144c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c146a90 con 0x7f660c0af4d0 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39840/0 (socket says 172.21.15.114:39840) 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 -- 172.21.15.114:0/2415764050 learned_addr learned my addr 172.21.15.114:0/2415764050 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612262700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f660c06c000 0x7f660c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6611a61700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 -- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 msgr2=0x7f660c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c1551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 -- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f660c06c000 msgr2=0x7f660c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f660c06c000 0x7f660c146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 -- 172.21.15.114:0/2415764050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc009550 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6612a63700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6604007e60 tx=0x7f660400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400ed30 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604014420 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040147d0 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c157f30 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.646+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c1584f0 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.647+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c0753f0 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.648+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6604014930 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.648+0000 7f6602ffd700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 0x7f65f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.648+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6604089b20 con 0x7f660c15e080 2023-11-24T06:25:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.648+0000 7f6612262700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 0x7f65f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.648+0000 7f6612262700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 0x7f65f006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000c00 tx=0x7f65fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.652+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6604054d10 con 0x7f660c15e080 2023-11-24T06:25:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.889+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f660c068670 con 0x7f660c15e080 2023-11-24T06:25:21.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.889+0000 7f6602ffd700 1 -- 172.21.15.114:0/2415764050 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f6604058330 con 0x7f660c15e080 2023-11-24T06:25:21.893 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:21.893 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:21.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 msgr2=0x7f65f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 0x7f65f006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000c00 tx=0x7f65fc008030 comp rx=0 tx=0).stop 2023-11-24T06:25:21.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 msgr2=0x7f660c155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6604007e60 tx=0x7f660400e3c0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 shutdown_connections 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006aed0 0x7f65f006d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f660c15e080 0x7f660c155700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f660c06c000 0x7f660c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 --2- 172.21.15.114:0/2415764050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660c0af4d0 0x7f660c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 >> 172.21.15.114:0/2415764050 conn(0x7f660c071b90 msgr2=0x7f660c0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 shutdown_connections 2023-11-24T06:25:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:21.892+0000 7f66144c6700 1 -- 172.21.15.114:0/2415764050 wait complete. 2023-11-24T06:25:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2023-11-24T06:25:21.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.024+0000 7f1db7a30700 1 Processor -- start 2023-11-24T06:25:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.024+0000 7f1db7a30700 1 -- start start 2023-11-24T06:25:22.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db015ddd0 0x7f1db00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1db015e1b0 0x7f1db00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1db00d1b30 0x7f1db00d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00d2500 con 0x7f1db015e1b0 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00b1d40 con 0x7f1db00d1b30 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db7a30700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b28c0 con 0x7f1db015ddd0 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1db015e1b0 0x7f1db00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1db015e1b0 0x7f1db00cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39846/0 (socket says 172.21.15.114:39846) 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3583251545 learned_addr learned my addr 172.21.15.114:0/3583251545 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db5fcd700 1 --2- 172.21.15.114:0/3583251545 >> v2:172.21.15.172:3300/0 conn(0x7f1db00d1b30 0x7f1db00d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db57cc700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db015ddd0 0x7f1db00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3583251545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db015ddd0 msgr2=0x7f1db00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db015ddd0 0x7f1db00b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3583251545 >> v2:172.21.15.172:3300/0 conn(0x7f1db00d1b30 msgr2=0x7f1db00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3583251545 >> v2:172.21.15.172:3300/0 conn(0x7f1db00d1b30 0x7f1db00d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3583251545 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00b1f90 con 0x7f1db015e1b0 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.025+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015e1b0 0x7f1db00cf000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b2830 tx=0x7f1da0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21110635454bd2b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1da7fff700 1 -- 172.21.15.114:0/3583251545 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011830 con 0x7f1db015e1b0 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1da7fff700 1 -- 172.21.15.114:0/3583251545 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0011990 con 0x7f1db015e1b0 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1da7fff700 1 -- 172.21.15.114:0/3583251545 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011d20 con 0x7f1db015e1b0 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 -- 172.21.15.114:0/3583251545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015e1b0 msgr2=0x7f1db00cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015e1b0 0x7f1db00cf000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b2830 tx=0x7f1da0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 -- 172.21.15.114:0/3583251545 shutdown_connections 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db015ddd0 0x7f1db00b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3583251545 >> v2:172.21.15.172:3300/0 conn(0x7f1db00d1b30 0x7f1db00d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3583251545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015e1b0 0x7f1db00cf000 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 -- 172.21.15.114:0/3583251545 >> 172.21.15.114:0/3583251545 conn(0x7f1db00731a0 msgr2=0x7f1db009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 -- 172.21.15.114:0/3583251545 shutdown_connections 2023-11-24T06:25:22.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.026+0000 7f1db7a30700 1 -- 172.21.15.114:0/3583251545 wait complete. 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.027+0000 7f1db7a30700 1 Processor -- start 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.027+0000 7f1db7a30700 1 -- start start 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db00d1b30 0x7f1db01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1db015e1b0 0x7f1db015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db009dc30 con 0x7f1db015ddd0 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db009cfb0 con 0x7f1db015e1b0 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db7a30700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db009d290 con 0x7f1db00d1b30 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db4fcb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db4fcb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39852/0 (socket says 172.21.15.114:39852) 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3981256573 learned_addr learned my addr 172.21.15.114:0/3981256573 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db57cc700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db00d1b30 0x7f1db01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.028+0000 7f1db5fcd700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1db015e1b0 0x7f1db015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db00d1b30 msgr2=0x7f1db01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db00d1b30 0x7f1db01545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1db015e1b0 msgr2=0x7f1db015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1db015e1b0 0x7f1db015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 -- 172.21.15.114:0/3981256573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009550 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db4fcb700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004240 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da001f070 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db00b2ed0 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00036a0 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.029+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db009d850 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.030+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db0067f60 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.030+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1da0021030 con 0x7f1db015ddd0 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.031+0000 7f1da5ffb700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 0x7f1d9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.031+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1da008a7c0 con 0x7f1db015ddd0 2023-11-24T06:25:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.031+0000 7f1db57cc700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 0x7f1d9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.031+0000 7f1db57cc700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 0x7f1d9406d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009be0 tx=0x7f1dac009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.035+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1da0055860 con 0x7f1db015ddd0 2023-11-24T06:25:22.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.227+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1db00b7c60 con 0x7f1db015ddd0 2023-11-24T06:25:22.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.228+0000 7f1da5ffb700 1 -- 172.21.15.114:0/3981256573 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f1da0058e80 con 0x7f1db015ddd0 2023-11-24T06:25:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:22.232 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:22.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.230+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 msgr2=0x7f1d9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.230+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 0x7f1d9406d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009be0 tx=0x7f1dac009210 comp rx=0 tx=0).stop 2023-11-24T06:25:22.234 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.230+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 msgr2=0x7f1db015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.230+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0004750 comp rx=0 tx=0).stop 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 shutdown_connections 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1d9406aed0 0x7f1d9406d390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1db015e1b0 0x7f1db015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1db00d1b30 0x7f1db01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 --2- 172.21.15.114:0/3981256573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1db015ddd0 0x7f1db015f710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 >> 172.21.15.114:0/3981256573 conn(0x7f1db00731a0 msgr2=0x7f1db006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 shutdown_connections 2023-11-24T06:25:22.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.231+0000 7f1db7a30700 1 -- 172.21.15.114:0/3981256573 wait complete. 2023-11-24T06:25:22.246 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:25:22.247 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:22.348 INFO:tasks.ceph.mds.c.smithi114.stderr:2023-11-24T06:25:22.344+0000 7f6fdfbb4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:25:22.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.363+0000 7f5d7a903700 1 Processor -- start 2023-11-24T06:25:22.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.363+0000 7f5d7a903700 1 -- start start 2023-11-24T06:25:22.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d7a903700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d740af4d0 0x7f5d740b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d7a903700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d740ae430 con 0x7f5d7406c710 2023-11-24T06:25:22.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d7a903700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d740ae710 con 0x7f5d7415e080 2023-11-24T06:25:22.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d7a903700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d740af0b0 con 0x7f5d740af4d0 2023-11-24T06:25:22.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d78ea0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d740af4d0 0x7f5d740b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d737fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5d7406c710 0x7f5d7406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55568/0 (socket says 172.21.15.114:55568) 2023-11-24T06:25:22.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d737fe700 1 -- 172.21.15.114:0/3380584696 learned_addr learned my addr 172.21.15.114:0/3380584696 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:22.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d78ea0700 1 -- 172.21.15.114:0/3380584696 >> v1:172.21.15.172:6789/0 conn(0x7f5d7415e080 legacy=0x7f5d7406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d78ea0700 1 -- 172.21.15.114:0/3380584696 >> v1:172.21.15.114:6789/0 conn(0x7f5d7406c710 legacy=0x7f5d7406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d72ffd700 1 -- 172.21.15.114:0/3380584696 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028690271 0 0) 0x7f5d740ae710 con 0x7f5d7415e080 2023-11-24T06:25:22.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d78ea0700 1 -- 172.21.15.114:0/3380584696 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740ae9f0 con 0x7f5d740af4d0 2023-11-24T06:25:22.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.364+0000 7f5d78ea0700 1 --2- 172.21.15.114:0/3380584696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d740af4d0 0x7f5d740b5050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009d90 tx=0x7f5d6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15b2135865950ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d72ffd700 1 -- 172.21.15.114:0/3380584696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00e070 con 0x7f5d740af4d0 2023-11-24T06:25:22.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d72ffd700 1 -- 172.21.15.114:0/3380584696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00bad0 con 0x7f5d740af4d0 2023-11-24T06:25:22.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d72ffd700 1 -- 172.21.15.114:0/3380584696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00be80 con 0x7f5d740af4d0 2023-11-24T06:25:22.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 -- 172.21.15.114:0/3380584696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d740af4d0 msgr2=0x7f5d740b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 --2- 172.21.15.114:0/3380584696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d740af4d0 0x7f5d740b5050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009d90 tx=0x7f5d6c00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:22.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 -- 172.21.15.114:0/3380584696 shutdown_connections 2023-11-24T06:25:22.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 --2- 172.21.15.114:0/3380584696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d740af4d0 0x7f5d740b5050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 -- 172.21.15.114:0/3380584696 >> 172.21.15.114:0/3380584696 conn(0x7f5d74071b90 msgr2=0x7f5d74071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 -- 172.21.15.114:0/3380584696 shutdown_connections 2023-11-24T06:25:22.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.365+0000 7f5d7a903700 1 -- 172.21.15.114:0/3380584696 wait complete. 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.366+0000 7f5d7a903700 1 Processor -- start 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 -- start start 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d740af4d0 0x7f5d740cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d7415e080 0x7f5d740cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74142f90 con 0x7f5d7406c710 2023-11-24T06:25:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74145bd0 con 0x7f5d7415e080 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d7a903700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74143250 con 0x7f5d740af4d0 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d73fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d73fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39864/0 (socket says 172.21.15.114:39864) 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d73fff700 1 -- 172.21.15.114:0/2446931359 learned_addr learned my addr 172.21.15.114:0/2446931359 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d737fe700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d740af4d0 0x7f5d740cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.367+0000 7f5d78ea0700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d7415e080 0x7f5d740cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 -- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d740af4d0 msgr2=0x7f5d740cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d740af4d0 0x7f5d740cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 -- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d7415e080 msgr2=0x7f5d740cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d7415e080 0x7f5d740cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 -- 172.21.15.114:0/2446931359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c009a70 con 0x7f5d7406c710 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d73fff700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800c550 tx=0x7f5d6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680054e0 con 0x7f5d7406c710 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68003680 con 0x7f5d7406c710 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68003a30 con 0x7f5d7406c710 2023-11-24T06:25:22.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d74143530 con 0x7f5d7406c710 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.368+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d74144ea0 con 0x7f5d7406c710 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.369+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d740753f0 con 0x7f5d7406c710 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.369+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d68003b90 con 0x7f5d7406c710 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.370+0000 7f5d70ff9700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 0x7f5d6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.370+0000 7f5d737fe700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 0x7f5d6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.370+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d68088a80 con 0x7f5d7406c710 2023-11-24T06:25:22.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.370+0000 7f5d737fe700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 0x7f5d6006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d64007c20 tx=0x7f5d64007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.374+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d68053b20 con 0x7f5d7406c710 2023-11-24T06:25:22.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.566+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d74142e10 con 0x7f5d6006aed0 2023-11-24T06:25:22.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.574+0000 7f5d70ff9700 1 -- 172.21.15.114:0/2446931359 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288819 (secure 0 0 0) 0x7f5d64004920 con 0x7f5d6006aed0 2023-11-24T06:25:22.581 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:22.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 msgr2=0x7f5d6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 0x7f5d6006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d64007c20 tx=0x7f5d64007d70 comp rx=0 tx=0).stop 2023-11-24T06:25:22.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 msgr2=0x7f5d74142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800c550 tx=0x7f5d6800c910 comp rx=0 tx=0).stop 2023-11-24T06:25:22.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 shutdown_connections 2023-11-24T06:25:22.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d6006aed0 0x7f5d6006d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d7415e080 0x7f5d740cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d7406c710 0x7f5d74142510 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 --2- 172.21.15.114:0/2446931359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d740af4d0 0x7f5d740cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 >> 172.21.15.114:0/2446931359 conn(0x7f5d74071b90 msgr2=0x7f5d740b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 shutdown_connections 2023-11-24T06:25:22.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.578+0000 7f5d7a903700 1 -- 172.21.15.114:0/2446931359 wait complete. 2023-11-24T06:25:22.587 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:22.593 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-11-24T06:25:21.719996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220356,"kb_used_data":4160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754364,"statfs":{"total":773094113280,"available":772868468736,"internally_reserved":0,"allocated":4259840,"data_stored":2357796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.256041"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500950+0000","last_change":"2023-11-24T06:25:18.502262+0000","last_active":"2023-11-24T06:25:19.500950+0000","last_peered":"2023-11-24T06:25:19.500950+0000","last_clean":"2023-11-24T06:25:19.500950+0000","last_became_active":"2023-11-24T06:25:18.501902+0000","last_became_peered":"2023-11-24T06:25:18.501902+0000","last_unstale":"2023-11-24T06:25:19.500950+0000","last_undegraded":"2023-11-24T06:25:19.500950+0000","last_fullsized":"2023-11-24T06:25:19.500950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:04.180393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499506+0000","last_change":"2023-11-24T06:25:18.514731+0000","last_active":"2023-11-24T06:25:19.499506+0000","last_peered":"2023-11-24T06:25:19.499506+0000","last_clean":"2023-11-24T06:25:19.499506+0000","last_became_active":"2023-11-24T06:25:18.514109+0000","last_became_peered":"2023-11-24T06:25:18.514109+0000","last_unstale":"2023-11-24T06:25:19.499506+0000","last_undegraded":"2023-11-24T06:25:19.499506+0000","last_fullsized":"2023-11-24T06:25:19.499506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:40.775365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511733+0000","last_change":"2023-11-24T06:25:18.509947+0000","last_active":"2023-11-24T06:25:18.511733+0000","last_peered":"2023-11-24T06:25:18.511733+0000","last_clean":"2023-11-24T06:25:18.511733+0000","last_became_active":"2023-11-24T06:25:18.509264+0000","last_became_peered":"2023-11-24T06:25:18.509264+0000","last_unstale":"2023-11-24T06:25:18.511733+0000","last_undegraded":"2023-11-24T06:25:18.511733+0000","last_fullsized":"2023-11-24T06:25:18.511733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:33.645797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639033+0000","last_change":"2023-11-24T06:25:18.501205+0000","last_active":"2023-11-24T06:25:19.639033+0000","last_peered":"2023-11-24T06:25:19.639033+0000","last_clean":"2023-11-24T06:25:19.639033+0000","last_became_active":"2023-11-24T06:25:18.500554+0000","last_became_peered":"2023-11-24T06:25:18.500554+0000","last_unstale":"2023-11-24T06:25:19.639033+0000","last_undegraded":"2023-11-24T06:25:19.639033+0000","last_fullsized":"2023-11-24T06:25:19.639033+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:17.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.516907+0000","last_change":"2023-11-24T06:25:18.511577+0000","last_active":"2023-11-24T06:25:18.516907+0000","last_peered":"2023-11-24T06:25:18.516907+0000","last_clean":"2023-11-24T06:25:18.516907+0000","last_became_active":"2023-11-24T06:25:18.507913+0000","last_became_peered":"2023-11-24T06:25:18.507913+0000","last_unstale":"2023-11-24T06:25:18.516907+0000","last_undegraded":"2023-11-24T06:25:18.516907+0000","last_fullsized":"2023-11-24T06:25:18.516907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:19.554771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499391+0000","last_change":"2023-11-24T06:25:18.517349+0000","last_active":"2023-11-24T06:25:19.499391+0000","last_peered":"2023-11-24T06:25:19.499391+0000","last_clean":"2023-11-24T06:25:19.499391+0000","last_became_active":"2023-11-24T06:25:18.517141+0000","last_became_peered":"2023-11-24T06:25:18.517141+0000","last_unstale":"2023-11-24T06:25:19.499391+0000","last_undegraded":"2023-11-24T06:25:19.499391+0000","last_fullsized":"2023-11-24T06:25:19.499391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:24.205989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500588+0000","last_change":"2023-11-24T06:25:18.504344+0000","last_active":"2023-11-24T06:25:19.500588+0000","last_peered":"2023-11-24T06:25:19.500588+0000","last_clean":"2023-11-24T06:25:19.500588+0000","last_became_active":"2023-11-24T06:25:18.504087+0000","last_became_peered":"2023-11-24T06:25:18.504087+0000","last_unstale":"2023-11-24T06:25:19.500588+0000","last_undegraded":"2023-11-24T06:25:19.500588+0000","last_fullsized":"2023-11-24T06:25:19.500588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:11.289535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500287+0000","last_change":"2023-11-24T06:25:18.494796+0000","last_active":"2023-11-24T06:25:19.500287+0000","last_peered":"2023-11-24T06:25:19.500287+0000","last_clean":"2023-11-24T06:25:19.500287+0000","last_became_active":"2023-11-24T06:25:18.494514+0000","last_became_peered":"2023-11-24T06:25:18.494514+0000","last_unstale":"2023-11-24T06:25:19.500287+0000","last_undegraded":"2023-11-24T06:25:19.500287+0000","last_fullsized":"2023-11-24T06:25:19.500287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:08.580752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.497546+0000","last_change":"2023-11-24T06:25:18.494885+0000","last_active":"2023-11-24T06:25:18.497546+0000","last_peered":"2023-11-24T06:25:18.497546+0000","last_clean":"2023-11-24T06:25:18.497546+0000","last_became_active":"2023-11-24T06:25:18.494604+0000","last_became_peered":"2023-11-24T06:25:18.494604+0000","last_unstale":"2023-11-24T06:25:18.497546+0000","last_undegraded":"2023-11-24T06:25:18.497546+0000","last_fullsized":"2023-11-24T06:25:18.497546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:21.873198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639215+0000","last_change":"2023-11-24T06:25:18.518929+0000","last_active":"2023-11-24T06:25:19.639215+0000","last_peered":"2023-11-24T06:25:19.639215+0000","last_clean":"2023-11-24T06:25:19.639215+0000","last_became_active":"2023-11-24T06:25:18.518663+0000","last_became_peered":"2023-11-24T06:25:18.518663+0000","last_unstale":"2023-11-24T06:25:19.639215+0000","last_undegraded":"2023-11-24T06:25:19.639215+0000","last_fullsized":"2023-11-24T06:25:19.639215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:24.512975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500622+0000","last_change":"2023-11-24T06:25:18.495155+0000","last_active":"2023-11-24T06:25:19.500622+0000","last_peered":"2023-11-24T06:25:19.500622+0000","last_clean":"2023-11-24T06:25:19.500622+0000","last_became_active":"2023-11-24T06:25:18.494831+0000","last_became_peered":"2023-11-24T06:25:18.494831+0000","last_unstale":"2023-11-24T06:25:19.500622+0000","last_undegraded":"2023-11-24T06:25:19.500622+0000","last_fullsized":"2023-11-24T06:25:19.500622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:23.861385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506856+0000","last_change":"2023-11-24T06:25:18.500859+0000","last_active":"2023-11-24T06:25:18.506856+0000","last_peered":"2023-11-24T06:25:18.506856+0000","last_clean":"2023-11-24T06:25:18.506856+0000","last_became_active":"2023-11-24T06:25:18.500586+0000","last_became_peered":"2023-11-24T06:25:18.500586+0000","last_unstale":"2023-11-24T06:25:18.506856+0000","last_undegraded":"2023-11-24T06:25:18.506856+0000","last_fullsized":"2023-11-24T06:25:18.506856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:14.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505326+0000","last_change":"2023-11-24T06:25:18.501559+0000","last_active":"2023-11-24T06:25:18.505326+0000","last_peered":"2023-11-24T06:25:18.505326+0000","last_clean":"2023-11-24T06:25:18.505326+0000","last_became_active":"2023-11-24T06:25:18.501275+0000","last_became_peered":"2023-11-24T06:25:18.501275+0000","last_unstale":"2023-11-24T06:25:18.505326+0000","last_undegraded":"2023-11-24T06:25:18.505326+0000","last_fullsized":"2023-11-24T06:25:18.505326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:20.171874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499403+0000","last_change":"2023-11-24T06:25:18.512608+0000","last_active":"2023-11-24T06:25:19.499403+0000","last_peered":"2023-11-24T06:25:19.499403+0000","last_clean":"2023-11-24T06:25:19.499403+0000","last_became_active":"2023-11-24T06:25:18.507454+0000","last_became_peered":"2023-11-24T06:25:18.507454+0000","last_unstale":"2023-11-24T06:25:19.499403+0000","last_undegraded":"2023-11-24T06:25:19.499403+0000","last_fullsized":"2023-11-24T06:25:19.499403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.723713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507172+0000","last_change":"2023-11-24T06:25:18.505091+0000","last_active":"2023-11-24T06:25:18.507172+0000","last_peered":"2023-11-24T06:25:18.507172+0000","last_clean":"2023-11-24T06:25:18.507172+0000","last_became_active":"2023-11-24T06:25:18.504803+0000","last_became_peered":"2023-11-24T06:25:18.504803+0000","last_unstale":"2023-11-24T06:25:18.507172+0000","last_undegraded":"2023-11-24T06:25:18.507172+0000","last_fullsized":"2023-11-24T06:25:18.507172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:15.336037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500857+0000","last_change":"2023-11-24T06:25:18.500526+0000","last_active":"2023-11-24T06:25:19.500857+0000","last_peered":"2023-11-24T06:25:19.500857+0000","last_clean":"2023-11-24T06:25:19.500857+0000","last_became_active":"2023-11-24T06:25:18.500276+0000","last_became_peered":"2023-11-24T06:25:18.500276+0000","last_unstale":"2023-11-24T06:25:19.500857+0000","last_undegraded":"2023-11-24T06:25:19.500857+0000","last_fullsized":"2023-11-24T06:25:19.500857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:11.332932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501335+0000","last_change":"2023-11-24T06:25:18.503037+0000","last_active":"2023-11-24T06:25:19.501335+0000","last_peered":"2023-11-24T06:25:19.501335+0000","last_clean":"2023-11-24T06:25:19.501335+0000","last_became_active":"2023-11-24T06:25:18.502777+0000","last_became_peered":"2023-11-24T06:25:18.502777+0000","last_unstale":"2023-11-24T06:25:19.501335+0000","last_undegraded":"2023-11-24T06:25:19.501335+0000","last_fullsized":"2023-11-24T06:25:19.501335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:40.336914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499609+0000","last_change":"2023-11-24T06:25:18.515651+0000","last_active":"2023-11-24T06:25:19.499609+0000","last_peered":"2023-11-24T06:25:19.499609+0000","last_clean":"2023-11-24T06:25:19.499609+0000","last_became_active":"2023-11-24T06:25:18.514154+0000","last_became_peered":"2023-11-24T06:25:18.514154+0000","last_unstale":"2023-11-24T06:25:19.499609+0000","last_undegraded":"2023-11-24T06:25:19.499609+0000","last_fullsized":"2023-11-24T06:25:19.499609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:12.018354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499162+0000","last_change":"2023-11-24T06:25:18.516522+0000","last_active":"2023-11-24T06:25:19.499162+0000","last_peered":"2023-11-24T06:25:19.499162+0000","last_clean":"2023-11-24T06:25:19.499162+0000","last_became_active":"2023-11-24T06:25:18.514196+0000","last_became_peered":"2023-11-24T06:25:18.514196+0000","last_unstale":"2023-11-24T06:25:19.499162+0000","last_undegraded":"2023-11-24T06:25:19.499162+0000","last_fullsized":"2023-11-24T06:25:19.499162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:01.050108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505631+0000","last_change":"2023-11-24T06:25:18.499047+0000","last_active":"2023-11-24T06:25:18.505631+0000","last_peered":"2023-11-24T06:25:18.505631+0000","last_clean":"2023-11-24T06:25:18.505631+0000","last_became_active":"2023-11-24T06:25:18.498778+0000","last_became_peered":"2023-11-24T06:25:18.498778+0000","last_unstale":"2023-11-24T06:25:18.505631+0000","last_undegraded":"2023-11-24T06:25:18.505631+0000","last_fullsized":"2023-11-24T06:25:18.505631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:32.206913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507523+0000","last_change":"2023-11-24T06:25:18.500860+0000","last_active":"2023-11-24T06:25:18.507523+0000","last_peered":"2023-11-24T06:25:18.507523+0000","last_clean":"2023-11-24T06:25:18.507523+0000","last_became_active":"2023-11-24T06:25:18.500583+0000","last_became_peered":"2023-11-24T06:25:18.500583+0000","last_unstale":"2023-11-24T06:25:18.507523+0000","last_undegraded":"2023-11-24T06:25:18.507523+0000","last_fullsized":"2023-11-24T06:25:18.507523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:16.853872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507225+0000","last_change":"2023-11-24T06:25:18.502994+0000","last_active":"2023-11-24T06:25:18.507225+0000","last_peered":"2023-11-24T06:25:18.507225+0000","last_clean":"2023-11-24T06:25:18.507225+0000","last_became_active":"2023-11-24T06:25:18.502697+0000","last_became_peered":"2023-11-24T06:25:18.502697+0000","last_unstale":"2023-11-24T06:25:18.507225+0000","last_undegraded":"2023-11-24T06:25:18.507225+0000","last_fullsized":"2023-11-24T06:25:18.507225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:10.813108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688733+0000","last_change":"2023-11-24T06:25:18.502958+0000","last_active":"2023-11-24T06:25:19.688733+0000","last_peered":"2023-11-24T06:25:19.688733+0000","last_clean":"2023-11-24T06:25:19.688733+0000","last_became_active":"2023-11-24T06:25:18.502635+0000","last_became_peered":"2023-11-24T06:25:18.502635+0000","last_unstale":"2023-11-24T06:25:19.688733+0000","last_undegraded":"2023-11-24T06:25:19.688733+0000","last_fullsized":"2023-11-24T06:25:19.688733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:45.204085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.508457+0000","last_change":"2023-11-24T06:25:18.506042+0000","last_active":"2023-11-24T06:25:18.508457+0000","last_peered":"2023-11-24T06:25:18.508457+0000","last_clean":"2023-11-24T06:25:18.508457+0000","last_became_active":"2023-11-24T06:25:18.505781+0000","last_became_peered":"2023-11-24T06:25:18.505781+0000","last_unstale":"2023-11-24T06:25:18.508457+0000","last_undegraded":"2023-11-24T06:25:18.508457+0000","last_fullsized":"2023-11-24T06:25:18.508457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:45.204085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639307+0000","last_change":"2023-11-24T06:25:18.501979+0000","last_active":"2023-11-24T06:25:19.639307+0000","last_peered":"2023-11-24T06:25:19.639307+0000","last_clean":"2023-11-24T06:25:19.639307+0000","last_became_active":"2023-11-24T06:25:18.500876+0000","last_became_peered":"2023-11-24T06:25:18.500876+0000","last_unstale":"2023-11-24T06:25:19.639307+0000","last_undegraded":"2023-11-24T06:25:19.639307+0000","last_fullsized":"2023-11-24T06:25:19.639307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:23.861385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499632+0000","last_change":"2023-11-24T06:25:18.515922+0000","last_active":"2023-11-24T06:25:19.499632+0000","last_peered":"2023-11-24T06:25:19.499632+0000","last_clean":"2023-11-24T06:25:19.499632+0000","last_became_active":"2023-11-24T06:25:18.514414+0000","last_became_peered":"2023-11-24T06:25:18.514414+0000","last_unstale":"2023-11-24T06:25:19.499632+0000","last_undegraded":"2023-11-24T06:25:19.499632+0000","last_fullsized":"2023-11-24T06:25:19.499632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:19.331758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639217+0000","last_change":"2023-11-24T06:25:18.509473+0000","last_active":"2023-11-24T06:25:19.639217+0000","last_peered":"2023-11-24T06:25:19.639217+0000","last_clean":"2023-11-24T06:25:19.639217+0000","last_became_active":"2023-11-24T06:25:18.508623+0000","last_became_peered":"2023-11-24T06:25:18.508623+0000","last_unstale":"2023-11-24T06:25:19.639217+0000","last_undegraded":"2023-11-24T06:25:19.639217+0000","last_fullsized":"2023-11-24T06:25:19.639217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:51.733991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506873+0000","last_change":"2023-11-24T06:25:18.504009+0000","last_active":"2023-11-24T06:25:18.506873+0000","last_peered":"2023-11-24T06:25:18.506873+0000","last_clean":"2023-11-24T06:25:18.506873+0000","last_became_active":"2023-11-24T06:25:18.503112+0000","last_became_peered":"2023-11-24T06:25:18.503112+0000","last_unstale":"2023-11-24T06:25:18.506873+0000","last_undegraded":"2023-11-24T06:25:18.506873+0000","last_fullsized":"2023-11-24T06:25:18.506873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:04.180393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510045+0000","last_change":"2023-11-24T06:25:18.508249+0000","last_active":"2023-11-24T06:25:18.510045+0000","last_peered":"2023-11-24T06:25:18.510045+0000","last_clean":"2023-11-24T06:25:18.510045+0000","last_became_active":"2023-11-24T06:25:18.507964+0000","last_became_peered":"2023-11-24T06:25:18.507964+0000","last_unstale":"2023-11-24T06:25:18.510045+0000","last_undegraded":"2023-11-24T06:25:18.510045+0000","last_fullsized":"2023-11-24T06:25:18.510045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:18.757773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500415+0000","last_change":"2023-11-24T06:25:18.502562+0000","last_active":"2023-11-24T06:25:19.500415+0000","last_peered":"2023-11-24T06:25:19.500415+0000","last_clean":"2023-11-24T06:25:19.500415+0000","last_became_active":"2023-11-24T06:25:18.501650+0000","last_became_peered":"2023-11-24T06:25:18.501650+0000","last_unstale":"2023-11-24T06:25:19.500415+0000","last_undegraded":"2023-11-24T06:25:19.500415+0000","last_fullsized":"2023-11-24T06:25:19.500415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:24.512975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500632+0000","last_change":"2023-11-24T06:25:18.509286+0000","last_active":"2023-11-24T06:25:19.500632+0000","last_peered":"2023-11-24T06:25:19.500632+0000","last_clean":"2023-11-24T06:25:19.500632+0000","last_became_active":"2023-11-24T06:25:18.508803+0000","last_became_peered":"2023-11-24T06:25:18.508803+0000","last_unstale":"2023-11-24T06:25:19.500632+0000","last_undegraded":"2023-11-24T06:25:19.500632+0000","last_fullsized":"2023-11-24T06:25:19.500632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:26:17.506990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501204+0000","last_change":"2023-11-24T06:25:18.499234+0000","last_active":"2023-11-24T06:25:19.501204+0000","last_peered":"2023-11-24T06:25:19.501204+0000","last_clean":"2023-11-24T06:25:19.501204+0000","last_became_active":"2023-11-24T06:25:18.498958+0000","last_became_peered":"2023-11-24T06:25:18.498958+0000","last_unstale":"2023-11-24T06:25:19.501204+0000","last_undegraded":"2023-11-24T06:25:19.501204+0000","last_fullsized":"2023-11-24T06:25:19.501204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.723713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688839+0000","last_change":"2023-11-24T06:25:18.503692+0000","last_active":"2023-11-24T06:25:19.688839+0000","last_peered":"2023-11-24T06:25:19.688839+0000","last_clean":"2023-11-24T06:25:19.688839+0000","last_became_active":"2023-11-24T06:25:18.503381+0000","last_became_peered":"2023-11-24T06:25:18.503381+0000","last_unstale":"2023-11-24T06:25:19.688839+0000","last_undegraded":"2023-11-24T06:25:19.688839+0000","last_fullsized":"2023-11-24T06:25:19.688839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:14.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501171+0000","last_change":"2023-11-24T06:25:17.488220+0000","last_active":"2023-11-24T06:25:19.501171+0000","last_peered":"2023-11-24T06:25:19.501171+0000","last_clean":"2023-11-24T06:25:19.501171+0000","last_became_active":"2023-11-24T06:25:17.487826+0000","last_became_peered":"2023-11-24T06:25:17.487826+0000","last_unstale":"2023-11-24T06:25:19.501171+0000","last_undegraded":"2023-11-24T06:25:19.501171+0000","last_fullsized":"2023-11-24T06:25:19.501171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:25.824047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501109+0000","last_change":"2023-11-24T06:25:18.501338+0000","last_active":"2023-11-24T06:25:19.501109+0000","last_peered":"2023-11-24T06:25:19.501109+0000","last_clean":"2023-11-24T06:25:19.501109+0000","last_became_active":"2023-11-24T06:25:18.501008+0000","last_became_peered":"2023-11-24T06:25:18.501008+0000","last_unstale":"2023-11-24T06:25:19.501109+0000","last_undegraded":"2023-11-24T06:25:19.501109+0000","last_fullsized":"2023-11-24T06:25:19.501109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:08.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688890+0000","last_change":"2023-11-24T06:25:17.486512+0000","last_active":"2023-11-24T06:25:19.688890+0000","last_peered":"2023-11-24T06:25:19.688890+0000","last_clean":"2023-11-24T06:25:19.688890+0000","last_became_active":"2023-11-24T06:25:17.486091+0000","last_became_peered":"2023-11-24T06:25:17.486091+0000","last_unstale":"2023-11-24T06:25:19.688890+0000","last_undegraded":"2023-11-24T06:25:19.688890+0000","last_fullsized":"2023-11-24T06:25:19.688890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:25.408123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510350+0000","last_change":"2023-11-24T06:25:18.508205+0000","last_active":"2023-11-24T06:25:18.510350+0000","last_peered":"2023-11-24T06:25:18.510350+0000","last_clean":"2023-11-24T06:25:18.510350+0000","last_became_active":"2023-11-24T06:25:18.507911+0000","last_became_peered":"2023-11-24T06:25:18.507911+0000","last_unstale":"2023-11-24T06:25:18.510350+0000","last_undegraded":"2023-11-24T06:25:18.510350+0000","last_fullsized":"2023-11-24T06:25:18.510350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:26:17.506990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499162+0000","last_change":"2023-11-24T06:25:17.485162+0000","last_active":"2023-11-24T06:25:19.499162+0000","last_peered":"2023-11-24T06:25:19.499162+0000","last_clean":"2023-11-24T06:25:19.499162+0000","last_became_active":"2023-11-24T06:25:17.484879+0000","last_became_peered":"2023-11-24T06:25:17.484879+0000","last_unstale":"2023-11-24T06:25:19.499162+0000","last_undegraded":"2023-11-24T06:25:19.499162+0000","last_fullsized":"2023-11-24T06:25:19.499162+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:22.851277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639294+0000","last_change":"2023-11-24T06:25:18.508950+0000","last_active":"2023-11-24T06:25:19.639294+0000","last_peered":"2023-11-24T06:25:19.639294+0000","last_clean":"2023-11-24T06:25:19.639294+0000","last_became_active":"2023-11-24T06:25:18.508429+0000","last_became_peered":"2023-11-24T06:25:18.508429+0000","last_unstale":"2023-11-24T06:25:19.639294+0000","last_undegraded":"2023-11-24T06:25:19.639294+0000","last_fullsized":"2023-11-24T06:25:19.639294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:11.289535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.514735+0000","last_change":"2023-11-24T06:25:17.486491+0000","last_active":"2023-11-24T06:25:18.514735+0000","last_peered":"2023-11-24T06:25:18.514735+0000","last_clean":"2023-11-24T06:25:18.514735+0000","last_became_active":"2023-11-24T06:25:17.486232+0000","last_became_peered":"2023-11-24T06:25:17.486232+0000","last_unstale":"2023-11-24T06:25:18.514735+0000","last_undegraded":"2023-11-24T06:25:18.514735+0000","last_fullsized":"2023-11-24T06:25:18.514735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:51.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500565+0000","last_change":"2023-11-24T06:25:18.496438+0000","last_active":"2023-11-24T06:25:19.500565+0000","last_peered":"2023-11-24T06:25:19.500565+0000","last_clean":"2023-11-24T06:25:19.500565+0000","last_became_active":"2023-11-24T06:25:18.496150+0000","last_became_peered":"2023-11-24T06:25:18.496150+0000","last_unstale":"2023-11-24T06:25:19.500565+0000","last_undegraded":"2023-11-24T06:25:19.500565+0000","last_fullsized":"2023-11-24T06:25:19.500565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:51.733991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.515927+0000","last_change":"2023-11-24T06:25:17.481913+0000","last_active":"2023-11-24T06:25:18.515927+0000","last_peered":"2023-11-24T06:25:18.515927+0000","last_clean":"2023-11-24T06:25:18.515927+0000","last_became_active":"2023-11-24T06:25:17.481634+0000","last_became_peered":"2023-11-24T06:25:17.481634+0000","last_unstale":"2023-11-24T06:25:18.515927+0000","last_undegraded":"2023-11-24T06:25:18.515927+0000","last_fullsized":"2023-11-24T06:25:18.515927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:34.526611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506854+0000","last_change":"2023-11-24T06:25:18.503501+0000","last_active":"2023-11-24T06:25:18.506854+0000","last_peered":"2023-11-24T06:25:18.506854+0000","last_clean":"2023-11-24T06:25:18.506854+0000","last_became_active":"2023-11-24T06:25:18.503030+0000","last_became_peered":"2023-11-24T06:25:18.503030+0000","last_unstale":"2023-11-24T06:25:18.506854+0000","last_undegraded":"2023-11-24T06:25:18.506854+0000","last_fullsized":"2023-11-24T06:25:18.506854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:08.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.518502+0000","last_change":"2023-11-24T06:25:17.482000+0000","last_active":"2023-11-24T06:25:18.518502+0000","last_peered":"2023-11-24T06:25:18.518502+0000","last_clean":"2023-11-24T06:25:18.518502+0000","last_became_active":"2023-11-24T06:25:17.481670+0000","last_became_peered":"2023-11-24T06:25:17.481670+0000","last_unstale":"2023-11-24T06:25:18.518502+0000","last_undegraded":"2023-11-24T06:25:18.518502+0000","last_fullsized":"2023-11-24T06:25:18.518502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:51.768193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506750+0000","last_change":"2023-11-24T06:25:18.500238+0000","last_active":"2023-11-24T06:25:18.506750+0000","last_peered":"2023-11-24T06:25:18.506750+0000","last_clean":"2023-11-24T06:25:18.506750+0000","last_became_active":"2023-11-24T06:25:18.499798+0000","last_became_peered":"2023-11-24T06:25:18.499798+0000","last_unstale":"2023-11-24T06:25:18.506750+0000","last_undegraded":"2023-11-24T06:25:18.506750+0000","last_fullsized":"2023-11-24T06:25:18.506750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:12.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498976+0000","last_change":"2023-11-24T06:25:17.481893+0000","last_active":"2023-11-24T06:25:19.498976+0000","last_peered":"2023-11-24T06:25:19.498976+0000","last_clean":"2023-11-24T06:25:19.498976+0000","last_became_active":"2023-11-24T06:25:17.481560+0000","last_became_peered":"2023-11-24T06:25:17.481560+0000","last_unstale":"2023-11-24T06:25:19.498976+0000","last_undegraded":"2023-11-24T06:25:19.498976+0000","last_fullsized":"2023-11-24T06:25:19.498976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:08.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638874+0000","last_change":"2023-11-24T06:25:18.497901+0000","last_active":"2023-11-24T06:25:19.638874+0000","last_peered":"2023-11-24T06:25:19.638874+0000","last_clean":"2023-11-24T06:25:19.638874+0000","last_became_active":"2023-11-24T06:25:18.497630+0000","last_became_peered":"2023-11-24T06:25:18.497630+0000","last_unstale":"2023-11-24T06:25:19.638874+0000","last_undegraded":"2023-11-24T06:25:19.638874+0000","last_fullsized":"2023-11-24T06:25:19.638874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:33.645797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.486168+0000","last_change":"2023-11-24T06:25:17.483963+0000","last_active":"2023-11-24T06:25:18.486168+0000","last_peered":"2023-11-24T06:25:18.486168+0000","last_clean":"2023-11-24T06:25:18.486168+0000","last_became_active":"2023-11-24T06:25:17.483566+0000","last_became_peered":"2023-11-24T06:25:17.483566+0000","last_unstale":"2023-11-24T06:25:18.486168+0000","last_undegraded":"2023-11-24T06:25:18.486168+0000","last_fullsized":"2023-11-24T06:25:18.486168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:03.924569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.517476+0000","last_change":"2023-11-24T06:25:18.516389+0000","last_active":"2023-11-24T06:25:18.517476+0000","last_peered":"2023-11-24T06:25:18.517476+0000","last_clean":"2023-11-24T06:25:18.517476+0000","last_became_active":"2023-11-24T06:25:18.516109+0000","last_became_peered":"2023-11-24T06:25:18.516109+0000","last_unstale":"2023-11-24T06:25:18.517476+0000","last_undegraded":"2023-11-24T06:25:18.517476+0000","last_fullsized":"2023-11-24T06:25:18.517476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:08.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499423+0000","last_change":"2023-11-24T06:25:17.486575+0000","last_active":"2023-11-24T06:25:19.499423+0000","last_peered":"2023-11-24T06:25:19.499423+0000","last_clean":"2023-11-24T06:25:19.499423+0000","last_became_active":"2023-11-24T06:25:17.486301+0000","last_became_peered":"2023-11-24T06:25:17.486301+0000","last_unstale":"2023-11-24T06:25:19.499423+0000","last_undegraded":"2023-11-24T06:25:19.499423+0000","last_fullsized":"2023-11-24T06:25:19.499423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:50.723882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499383+0000","last_change":"2023-11-24T06:25:18.516055+0000","last_active":"2023-11-24T06:25:19.499383+0000","last_peered":"2023-11-24T06:25:19.499383+0000","last_clean":"2023-11-24T06:25:19.499383+0000","last_became_active":"2023-11-24T06:25:18.515789+0000","last_became_peered":"2023-11-24T06:25:18.515789+0000","last_unstale":"2023-11-24T06:25:19.499383+0000","last_undegraded":"2023-11-24T06:25:19.499383+0000","last_fullsized":"2023-11-24T06:25:19.499383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:05.460194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688796+0000","last_change":"2023-11-24T06:25:17.482356+0000","last_active":"2023-11-24T06:25:19.688796+0000","last_peered":"2023-11-24T06:25:19.688796+0000","last_clean":"2023-11-24T06:25:19.688796+0000","last_became_active":"2023-11-24T06:25:17.482087+0000","last_became_peered":"2023-11-24T06:25:17.482087+0000","last_unstale":"2023-11-24T06:25:19.688796+0000","last_undegraded":"2023-11-24T06:25:19.688796+0000","last_fullsized":"2023-11-24T06:25:19.688796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:11.578791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499231+0000","last_change":"2023-11-24T06:25:18.514073+0000","last_active":"2023-11-24T06:25:19.499231+0000","last_peered":"2023-11-24T06:25:19.499231+0000","last_clean":"2023-11-24T06:25:19.499231+0000","last_became_active":"2023-11-24T06:25:18.513216+0000","last_became_peered":"2023-11-24T06:25:18.513216+0000","last_unstale":"2023-11-24T06:25:19.499231+0000","last_undegraded":"2023-11-24T06:25:19.499231+0000","last_fullsized":"2023-11-24T06:25:19.499231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:11.332932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510567+0000","last_change":"2023-11-24T06:25:17.477898+0000","last_active":"2023-11-24T06:25:18.510567+0000","last_peered":"2023-11-24T06:25:18.510567+0000","last_clean":"2023-11-24T06:25:18.510567+0000","last_became_active":"2023-11-24T06:25:17.477479+0000","last_became_peered":"2023-11-24T06:25:17.477479+0000","last_unstale":"2023-11-24T06:25:18.510567+0000","last_undegraded":"2023-11-24T06:25:18.510567+0000","last_fullsized":"2023-11-24T06:25:18.510567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:16.417587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511752+0000","last_change":"2023-11-24T06:25:18.510150+0000","last_active":"2023-11-24T06:25:18.511752+0000","last_peered":"2023-11-24T06:25:18.511752+0000","last_clean":"2023-11-24T06:25:18.511752+0000","last_became_active":"2023-11-24T06:25:18.509661+0000","last_became_peered":"2023-11-24T06:25:18.509661+0000","last_unstale":"2023-11-24T06:25:18.511752+0000","last_undegraded":"2023-11-24T06:25:18.511752+0000","last_fullsized":"2023-11-24T06:25:18.511752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:35.536720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499320+0000","last_change":"2023-11-24T06:25:17.486745+0000","last_active":"2023-11-24T06:25:19.499320+0000","last_peered":"2023-11-24T06:25:19.499320+0000","last_clean":"2023-11-24T06:25:19.499320+0000","last_became_active":"2023-11-24T06:25:17.486318+0000","last_became_peered":"2023-11-24T06:25:17.486318+0000","last_unstale":"2023-11-24T06:25:19.499320+0000","last_undegraded":"2023-11-24T06:25:19.499320+0000","last_fullsized":"2023-11-24T06:25:19.499320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.279426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638893+0000","last_change":"2023-11-24T06:25:18.497371+0000","last_active":"2023-11-24T06:25:19.638893+0000","last_peered":"2023-11-24T06:25:19.638893+0000","last_clean":"2023-11-24T06:25:19.638893+0000","last_became_active":"2023-11-24T06:25:18.497144+0000","last_became_peered":"2023-11-24T06:25:18.497144+0000","last_unstale":"2023-11-24T06:25:19.638893+0000","last_undegraded":"2023-11-24T06:25:19.638893+0000","last_fullsized":"2023-11-24T06:25:19.638893+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:09.769193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511835+0000","last_change":"2023-11-24T06:25:17.483866+0000","last_active":"2023-11-24T06:25:18.511835+0000","last_peered":"2023-11-24T06:25:18.511835+0000","last_clean":"2023-11-24T06:25:18.511835+0000","last_became_active":"2023-11-24T06:25:17.483588+0000","last_became_peered":"2023-11-24T06:25:17.483588+0000","last_unstale":"2023-11-24T06:25:18.511835+0000","last_undegraded":"2023-11-24T06:25:18.511835+0000","last_fullsized":"2023-11-24T06:25:18.511835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:03.170284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499337+0000","last_change":"2023-11-24T06:25:18.516869+0000","last_active":"2023-11-24T06:25:19.499337+0000","last_peered":"2023-11-24T06:25:19.499337+0000","last_clean":"2023-11-24T06:25:19.499337+0000","last_became_active":"2023-11-24T06:25:18.513211+0000","last_became_peered":"2023-11-24T06:25:18.513211+0000","last_unstale":"2023-11-24T06:25:19.499337+0000","last_undegraded":"2023-11-24T06:25:19.499337+0000","last_fullsized":"2023-11-24T06:25:19.499337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:40.336914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638986+0000","last_change":"2023-11-24T06:25:17.487172+0000","last_active":"2023-11-24T06:25:19.638986+0000","last_peered":"2023-11-24T06:25:19.638986+0000","last_clean":"2023-11-24T06:25:19.638986+0000","last_became_active":"2023-11-24T06:25:17.486795+0000","last_became_peered":"2023-11-24T06:25:17.486795+0000","last_unstale":"2023-11-24T06:25:19.638986+0000","last_undegraded":"2023-11-24T06:25:19.638986+0000","last_fullsized":"2023-11-24T06:25:19.638986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:29.274409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501079+0000","last_change":"2023-11-24T06:25:18.499346+0000","last_active":"2023-11-24T06:25:19.501079+0000","last_peered":"2023-11-24T06:25:19.501079+0000","last_clean":"2023-11-24T06:25:19.501079+0000","last_became_active":"2023-11-24T06:25:18.499062+0000","last_became_peered":"2023-11-24T06:25:18.499062+0000","last_unstale":"2023-11-24T06:25:19.501079+0000","last_undegraded":"2023-11-24T06:25:19.501079+0000","last_fullsized":"2023-11-24T06:25:19.501079+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:05.460194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499006+0000","last_change":"2023-11-24T06:25:17.485252+0000","last_active":"2023-11-24T06:25:19.499006+0000","last_peered":"2023-11-24T06:25:19.499006+0000","last_clean":"2023-11-24T06:25:19.499006+0000","last_became_active":"2023-11-24T06:25:17.484972+0000","last_became_peered":"2023-11-24T06:25:17.484972+0000","last_unstale":"2023-11-24T06:25:19.499006+0000","last_undegraded":"2023-11-24T06:25:19.499006+0000","last_fullsized":"2023-11-24T06:25:19.499006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:16.417587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500479+0000","last_change":"2023-11-24T06:25:18.501889+0000","last_active":"2023-11-24T06:25:19.500479+0000","last_peered":"2023-11-24T06:25:19.500479+0000","last_clean":"2023-11-24T06:25:19.500479+0000","last_became_active":"2023-11-24T06:25:18.501582+0000","last_became_peered":"2023-11-24T06:25:18.501582+0000","last_unstale":"2023-11-24T06:25:19.500479+0000","last_undegraded":"2023-11-24T06:25:19.500479+0000","last_fullsized":"2023-11-24T06:25:19.500479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:24.490953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499062+0000","last_change":"2023-11-24T06:25:17.485031+0000","last_active":"2023-11-24T06:25:19.499062+0000","last_peered":"2023-11-24T06:25:19.499062+0000","last_clean":"2023-11-24T06:25:19.499062+0000","last_became_active":"2023-11-24T06:25:17.484743+0000","last_became_peered":"2023-11-24T06:25:17.484743+0000","last_unstale":"2023-11-24T06:25:19.499062+0000","last_undegraded":"2023-11-24T06:25:19.499062+0000","last_fullsized":"2023-11-24T06:25:19.499062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:07.570644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.499940+0000","last_change":"2023-11-24T06:25:18.496924+0000","last_active":"2023-11-24T06:25:18.499940+0000","last_peered":"2023-11-24T06:25:18.499940+0000","last_clean":"2023-11-24T06:25:18.499940+0000","last_became_active":"2023-11-24T06:25:18.496649+0000","last_became_peered":"2023-11-24T06:25:18.496649+0000","last_unstale":"2023-11-24T06:25:18.499940+0000","last_undegraded":"2023-11-24T06:25:18.499940+0000","last_fullsized":"2023-11-24T06:25:18.499940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:05:01.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639364+0000","last_change":"2023-11-24T06:25:17.482303+0000","last_active":"2023-11-24T06:25:19.639364+0000","last_peered":"2023-11-24T06:25:19.639364+0000","last_clean":"2023-11-24T06:25:19.639364+0000","last_became_active":"2023-11-24T06:25:17.482023+0000","last_became_peered":"2023-11-24T06:25:17.482023+0000","last_unstale":"2023-11-24T06:25:19.639364+0000","last_undegraded":"2023-11-24T06:25:19.639364+0000","last_fullsized":"2023-11-24T06:25:19.639364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:39.041951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505999+0000","last_change":"2023-11-24T06:25:18.498456+0000","last_active":"2023-11-24T06:25:18.505999+0000","last_peered":"2023-11-24T06:25:18.505999+0000","last_clean":"2023-11-24T06:25:18.505999+0000","last_became_active":"2023-11-24T06:25:18.498100+0000","last_became_peered":"2023-11-24T06:25:18.498100+0000","last_unstale":"2023-11-24T06:25:18.505999+0000","last_undegraded":"2023-11-24T06:25:18.505999+0000","last_fullsized":"2023-11-24T06:25:18.505999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:40.052060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511607+0000","last_change":"2023-11-24T06:25:17.486832+0000","last_active":"2023-11-24T06:25:18.511607+0000","last_peered":"2023-11-24T06:25:18.511607+0000","last_clean":"2023-11-24T06:25:18.511607+0000","last_became_active":"2023-11-24T06:25:17.486484+0000","last_became_peered":"2023-11-24T06:25:17.486484+0000","last_unstale":"2023-11-24T06:25:18.511607+0000","last_undegraded":"2023-11-24T06:25:18.511607+0000","last_fullsized":"2023-11-24T06:25:18.511607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:11.578791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639102+0000","last_change":"2023-11-24T06:25:18.503624+0000","last_active":"2023-11-24T06:25:19.639102+0000","last_peered":"2023-11-24T06:25:19.639102+0000","last_clean":"2023-11-24T06:25:19.639102+0000","last_became_active":"2023-11-24T06:25:18.500947+0000","last_became_peered":"2023-11-24T06:25:18.500947+0000","last_unstale":"2023-11-24T06:25:19.639102+0000","last_undegraded":"2023-11-24T06:25:19.639102+0000","last_fullsized":"2023-11-24T06:25:19.639102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:08.580752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.521788+0000","last_change":"2023-11-24T06:25:17.481879+0000","last_active":"2023-11-24T06:25:18.521788+0000","last_peered":"2023-11-24T06:25:18.521788+0000","last_clean":"2023-11-24T06:25:18.521788+0000","last_became_active":"2023-11-24T06:25:17.481562+0000","last_became_peered":"2023-11-24T06:25:17.481562+0000","last_unstale":"2023-11-24T06:25:18.521788+0000","last_undegraded":"2023-11-24T06:25:18.521788+0000","last_fullsized":"2023-11-24T06:25:18.521788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:18:06.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505414+0000","last_change":"2023-11-24T06:25:18.498968+0000","last_active":"2023-11-24T06:25:18.505414+0000","last_peered":"2023-11-24T06:25:18.505414+0000","last_clean":"2023-11-24T06:25:18.505414+0000","last_became_active":"2023-11-24T06:25:18.498682+0000","last_became_peered":"2023-11-24T06:25:18.498682+0000","last_unstale":"2023-11-24T06:25:18.505414+0000","last_undegraded":"2023-11-24T06:25:18.505414+0000","last_fullsized":"2023-11-24T06:25:18.505414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:34:51.769406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488821+0000","last_change":"2023-11-24T06:25:17.484206+0000","last_active":"2023-11-24T06:25:18.488821+0000","last_peered":"2023-11-24T06:25:18.488821+0000","last_clean":"2023-11-24T06:25:18.488821+0000","last_became_active":"2023-11-24T06:25:17.483937+0000","last_became_peered":"2023-11-24T06:25:17.483937+0000","last_unstale":"2023-11-24T06:25:18.488821+0000","last_undegraded":"2023-11-24T06:25:18.488821+0000","last_fullsized":"2023-11-24T06:25:18.488821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:07.933626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505253+0000","last_change":"2023-11-24T06:25:18.494824+0000","last_active":"2023-11-24T06:25:18.505253+0000","last_peered":"2023-11-24T06:25:18.505253+0000","last_clean":"2023-11-24T06:25:18.505253+0000","last_became_active":"2023-11-24T06:25:18.494537+0000","last_became_peered":"2023-11-24T06:25:18.494537+0000","last_unstale":"2023-11-24T06:25:18.505253+0000","last_undegraded":"2023-11-24T06:25:18.505253+0000","last_fullsized":"2023-11-24T06:25:18.505253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:16.811522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688911+0000","last_change":"2023-11-24T06:25:17.487142+0000","last_active":"2023-11-24T06:25:19.688911+0000","last_peered":"2023-11-24T06:25:19.688911+0000","last_clean":"2023-11-24T06:25:19.688911+0000","last_became_active":"2023-11-24T06:25:17.486871+0000","last_became_peered":"2023-11-24T06:25:17.486871+0000","last_unstale":"2023-11-24T06:25:19.688911+0000","last_undegraded":"2023-11-24T06:25:19.688911+0000","last_fullsized":"2023-11-24T06:25:19.688911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:34.526611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511673+0000","last_change":"2023-11-24T06:25:18.504881+0000","last_active":"2023-11-24T06:25:18.511673+0000","last_peered":"2023-11-24T06:25:18.511673+0000","last_clean":"2023-11-24T06:25:18.511673+0000","last_became_active":"2023-11-24T06:25:18.504611+0000","last_became_peered":"2023-11-24T06:25:18.504611+0000","last_unstale":"2023-11-24T06:25:18.511673+0000","last_undegraded":"2023-11-24T06:25:18.511673+0000","last_fullsized":"2023-11-24T06:25:18.511673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:47.180225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501039+0000","last_change":"2023-11-24T06:25:17.486593+0000","last_active":"2023-11-24T06:25:19.501039+0000","last_peered":"2023-11-24T06:25:19.501039+0000","last_clean":"2023-11-24T06:25:19.501039+0000","last_became_active":"2023-11-24T06:25:17.486300+0000","last_became_peered":"2023-11-24T06:25:17.486300+0000","last_unstale":"2023-11-24T06:25:19.501039+0000","last_undegraded":"2023-11-24T06:25:19.501039+0000","last_fullsized":"2023-11-24T06:25:19.501039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:17.747664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511179+0000","last_change":"2023-11-24T06:25:18.504808+0000","last_active":"2023-11-24T06:25:18.511179+0000","last_peered":"2023-11-24T06:25:18.511179+0000","last_clean":"2023-11-24T06:25:18.511179+0000","last_became_active":"2023-11-24T06:25:18.504556+0000","last_became_peered":"2023-11-24T06:25:18.504556+0000","last_unstale":"2023-11-24T06:25:18.511179+0000","last_undegraded":"2023-11-24T06:25:18.511179+0000","last_fullsized":"2023-11-24T06:25:18.511179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:42.387098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488323+0000","last_change":"2023-11-24T06:25:17.484744+0000","last_active":"2023-11-24T06:25:18.488323+0000","last_peered":"2023-11-24T06:25:18.488323+0000","last_clean":"2023-11-24T06:25:18.488323+0000","last_became_active":"2023-11-24T06:25:17.484476+0000","last_became_peered":"2023-11-24T06:25:17.484476+0000","last_unstale":"2023-11-24T06:25:18.488323+0000","last_undegraded":"2023-11-24T06:25:18.488323+0000","last_fullsized":"2023-11-24T06:25:18.488323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:25.408123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500738+0000","last_change":"2023-11-24T06:25:18.504231+0000","last_active":"2023-11-24T06:25:19.500738+0000","last_peered":"2023-11-24T06:25:19.500738+0000","last_clean":"2023-11-24T06:25:19.500738+0000","last_became_active":"2023-11-24T06:25:18.503949+0000","last_became_peered":"2023-11-24T06:25:18.503949+0000","last_unstale":"2023-11-24T06:25:19.500738+0000","last_undegraded":"2023-11-24T06:25:19.500738+0000","last_fullsized":"2023-11-24T06:25:19.500738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:46.986918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.492074+0000","last_change":"2023-11-24T06:25:17.489327+0000","last_active":"2023-11-24T06:25:18.492074+0000","last_peered":"2023-11-24T06:25:18.492074+0000","last_clean":"2023-11-24T06:25:18.492074+0000","last_became_active":"2023-11-24T06:25:17.488936+0000","last_became_peered":"2023-11-24T06:25:17.488936+0000","last_unstale":"2023-11-24T06:25:18.492074+0000","last_undegraded":"2023-11-24T06:25:18.492074+0000","last_fullsized":"2023-11-24T06:25:18.492074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:08.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505660+0000","last_change":"2023-11-24T06:25:18.501849+0000","last_active":"2023-11-24T06:25:18.505660+0000","last_peered":"2023-11-24T06:25:18.505660+0000","last_clean":"2023-11-24T06:25:18.505660+0000","last_became_active":"2023-11-24T06:25:18.499304+0000","last_became_peered":"2023-11-24T06:25:18.499304+0000","last_unstale":"2023-11-24T06:25:18.505660+0000","last_undegraded":"2023-11-24T06:25:18.505660+0000","last_fullsized":"2023-11-24T06:25:18.505660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:12.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639273+0000","last_change":"2023-11-24T06:25:17.484655+0000","last_active":"2023-11-24T06:25:19.639273+0000","last_peered":"2023-11-24T06:25:19.639273+0000","last_clean":"2023-11-24T06:25:19.639273+0000","last_became_active":"2023-11-24T06:25:17.484386+0000","last_became_peered":"2023-11-24T06:25:17.484386+0000","last_unstale":"2023-11-24T06:25:19.639273+0000","last_undegraded":"2023-11-24T06:25:19.639273+0000","last_fullsized":"2023-11-24T06:25:19.639273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:13.344722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507035+0000","last_change":"2023-11-24T06:25:18.501694+0000","last_active":"2023-11-24T06:25:18.507035+0000","last_peered":"2023-11-24T06:25:18.507035+0000","last_clean":"2023-11-24T06:25:18.507035+0000","last_became_active":"2023-11-24T06:25:18.501428+0000","last_became_peered":"2023-11-24T06:25:18.501428+0000","last_unstale":"2023-11-24T06:25:18.507035+0000","last_undegraded":"2023-11-24T06:25:18.507035+0000","last_fullsized":"2023-11-24T06:25:18.507035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:30.284518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688842+0000","last_change":"2023-11-24T06:25:17.486342+0000","last_active":"2023-11-24T06:25:19.688842+0000","last_peered":"2023-11-24T06:25:19.688842+0000","last_clean":"2023-11-24T06:25:19.688842+0000","last_became_active":"2023-11-24T06:25:17.485721+0000","last_became_peered":"2023-11-24T06:25:17.485721+0000","last_unstale":"2023-11-24T06:25:19.688842+0000","last_undegraded":"2023-11-24T06:25:19.688842+0000","last_fullsized":"2023-11-24T06:25:19.688842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:25.824047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688804+0000","last_change":"2023-11-24T06:25:18.510255+0000","last_active":"2023-11-24T06:25:19.688804+0000","last_peered":"2023-11-24T06:25:19.688804+0000","last_clean":"2023-11-24T06:25:19.688804+0000","last_became_active":"2023-11-24T06:25:18.510013+0000","last_became_peered":"2023-11-24T06:25:18.510013+0000","last_unstale":"2023-11-24T06:25:19.688804+0000","last_undegraded":"2023-11-24T06:25:19.688804+0000","last_fullsized":"2023-11-24T06:25:19.688804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:30.284518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639122+0000","last_change":"2023-11-24T06:25:17.485977+0000","last_active":"2023-11-24T06:25:19.639122+0000","last_peered":"2023-11-24T06:25:19.639122+0000","last_clean":"2023-11-24T06:25:19.639122+0000","last_became_active":"2023-11-24T06:25:17.485582+0000","last_became_peered":"2023-11-24T06:25:17.485582+0000","last_unstale":"2023-11-24T06:25:19.639122+0000","last_undegraded":"2023-11-24T06:25:19.639122+0000","last_fullsized":"2023-11-24T06:25:19.639122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:07.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500944+0000","last_change":"2023-11-24T06:25:18.502625+0000","last_active":"2023-11-24T06:25:19.500944+0000","last_peered":"2023-11-24T06:25:19.500944+0000","last_clean":"2023-11-24T06:25:19.500944+0000","last_became_active":"2023-11-24T06:25:18.502254+0000","last_became_peered":"2023-11-24T06:25:18.502254+0000","last_unstale":"2023-11-24T06:25:19.500944+0000","last_undegraded":"2023-11-24T06:25:19.500944+0000","last_fullsized":"2023-11-24T06:25:19.500944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:35.536720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688856+0000","last_change":"2023-11-24T06:25:17.487180+0000","last_active":"2023-11-24T06:25:19.688856+0000","last_peered":"2023-11-24T06:25:19.688856+0000","last_clean":"2023-11-24T06:25:19.688856+0000","last_became_active":"2023-11-24T06:25:17.486936+0000","last_became_peered":"2023-11-24T06:25:17.486936+0000","last_unstale":"2023-11-24T06:25:19.688856+0000","last_undegraded":"2023-11-24T06:25:19.688856+0000","last_fullsized":"2023-11-24T06:25:19.688856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:03.170284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688988+0000","last_change":"2023-11-24T06:25:18.502766+0000","last_active":"2023-11-24T06:25:19.688988+0000","last_peered":"2023-11-24T06:25:19.688988+0000","last_clean":"2023-11-24T06:25:19.688988+0000","last_became_active":"2023-11-24T06:25:18.502457+0000","last_became_peered":"2023-11-24T06:25:18.502457+0000","last_unstale":"2023-11-24T06:25:19.688988+0000","last_undegraded":"2023-11-24T06:25:19.688988+0000","last_fullsized":"2023-11-24T06:25:19.688988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:08.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638894+0000","last_change":"2023-11-24T06:25:17.484634+0000","last_active":"2023-11-24T06:25:19.638894+0000","last_peered":"2023-11-24T06:25:19.638894+0000","last_clean":"2023-11-24T06:25:19.638894+0000","last_became_active":"2023-11-24T06:25:17.484270+0000","last_became_peered":"2023-11-24T06:25:17.484270+0000","last_unstale":"2023-11-24T06:25:19.638894+0000","last_undegraded":"2023-11-24T06:25:19.638894+0000","last_fullsized":"2023-11-24T06:25:19.638894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:44.193976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498969+0000","last_change":"2023-11-24T06:25:18.519424+0000","last_active":"2023-11-24T06:25:19.498969+0000","last_peered":"2023-11-24T06:25:19.498969+0000","last_clean":"2023-11-24T06:25:19.498969+0000","last_became_active":"2023-11-24T06:25:18.519211+0000","last_became_peered":"2023-11-24T06:25:18.519211+0000","last_unstale":"2023-11-24T06:25:19.498969+0000","last_undegraded":"2023-11-24T06:25:19.498969+0000","last_fullsized":"2023-11-24T06:25:19.498969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:46.986918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488471+0000","last_change":"2023-11-24T06:25:17.487213+0000","last_active":"2023-11-24T06:25:18.488471+0000","last_peered":"2023-11-24T06:25:18.488471+0000","last_clean":"2023-11-24T06:25:18.488471+0000","last_became_active":"2023-11-24T06:25:17.486855+0000","last_became_peered":"2023-11-24T06:25:17.486855+0000","last_unstale":"2023-11-24T06:25:18.488471+0000","last_undegraded":"2023-11-24T06:25:18.488471+0000","last_fullsized":"2023-11-24T06:25:18.488471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:23.480844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.517623+0000","last_change":"2023-11-24T06:25:18.516471+0000","last_active":"2023-11-24T06:25:18.517623+0000","last_peered":"2023-11-24T06:25:18.517623+0000","last_clean":"2023-11-24T06:25:18.517623+0000","last_became_active":"2023-11-24T06:25:18.516153+0000","last_became_peered":"2023-11-24T06:25:18.516153+0000","last_unstale":"2023-11-24T06:25:18.517623+0000","last_undegraded":"2023-11-24T06:25:18.517623+0000","last_fullsized":"2023-11-24T06:25:18.517623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:26.834156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498994+0000","last_change":"2023-11-24T06:25:17.489308+0000","last_active":"2023-11-24T06:25:19.498994+0000","last_peered":"2023-11-24T06:25:19.498994+0000","last_clean":"2023-11-24T06:25:19.498994+0000","last_became_active":"2023-11-24T06:25:17.488994+0000","last_became_peered":"2023-11-24T06:25:17.488994+0000","last_unstale":"2023-11-24T06:25:19.498994+0000","last_undegraded":"2023-11-24T06:25:19.498994+0000","last_fullsized":"2023-11-24T06:25:19.498994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:14.325928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500833+0000","last_change":"2023-11-24T06:25:18.495007+0000","last_active":"2023-11-24T06:25:19.500833+0000","last_peered":"2023-11-24T06:25:19.500833+0000","last_clean":"2023-11-24T06:25:19.500833+0000","last_became_active":"2023-11-24T06:25:18.493873+0000","last_became_peered":"2023-11-24T06:25:18.493873+0000","last_unstale":"2023-11-24T06:25:19.500833+0000","last_undegraded":"2023-11-24T06:25:19.500833+0000","last_fullsized":"2023-11-24T06:25:19.500833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:01.050108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500344+0000","last_change":"2023-11-24T06:25:17.488128+0000","last_active":"2023-11-24T06:25:19.500344+0000","last_peered":"2023-11-24T06:25:19.500344+0000","last_clean":"2023-11-24T06:25:19.500344+0000","last_became_active":"2023-11-24T06:25:17.487844+0000","last_became_peered":"2023-11-24T06:25:17.487844+0000","last_unstale":"2023-11-24T06:25:19.500344+0000","last_undegraded":"2023-11-24T06:25:19.500344+0000","last_fullsized":"2023-11-24T06:25:19.500344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:09.802999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.487813+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:18.487813+0000","last_peered":"2023-11-24T06:25:18.487813+0000","last_clean":"2023-11-24T06:25:18.487813+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:18.487813+0000","last_undegraded":"2023-11-24T06:25:18.487813+0000","last_fullsized":"2023-11-24T06:25:18.487813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.438},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.505}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.79600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.249},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.622},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.5}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.621}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.434},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.374},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.32700000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:22.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:22.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.751+0000 7fadea6c9700 1 Processor -- start 2023-11-24T06:25:22.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.751+0000 7fadea6c9700 1 -- start start 2023-11-24T06:25:22.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fadea6c9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fade40b3370 0x7fade41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fadea6c9700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40b22d0 con 0x7fade415bdd0 2023-11-24T06:25:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fadea6c9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40b25b0 con 0x7fade40d1c70 2023-11-24T06:25:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fadea6c9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40b2f50 con 0x7fade40b3370 2023-11-24T06:25:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade3fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fade415bdd0 0x7fade40b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55580/0 (socket says 172.21.15.114:55580) 2023-11-24T06:25:22.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade3fff700 1 -- 172.21.15.114:0/1317606023 learned_addr learned my addr 172.21.15.114:0/1317606023 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:22.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656136639 0 0) 0x7fade40b22d0 con 0x7fade415bdd0 2023-11-24T06:25:22.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc0035b0 con 0x7fade415bdd0 2023-11-24T06:25:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade37fe700 1 --2- 172.21.15.114:0/1317606023 >> v2:172.21.15.172:3300/0 conn(0x7fade40b3370 0x7fade41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734188987 0 0) 0x7fadcc0035b0 con 0x7fade415bdd0 2023-11-24T06:25:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 >> v2:172.21.15.172:3300/0 conn(0x7fade40b3370 msgr2=0x7fade41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 --2- 172.21.15.114:0/1317606023 >> v2:172.21.15.172:3300/0 conn(0x7fade40b3370 0x7fade41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 >> v1:172.21.15.172:6789/0 conn(0x7fade40d1c70 legacy=0x7fade4165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40b2890 con 0x7fade415bdd0 2023-11-24T06:25:22.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fadd4001670 con 0x7fade415bdd0 2023-11-24T06:25:22.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 >> v1:172.21.15.114:6789/0 conn(0x7fade415bdd0 legacy=0x7fade40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc003a00 0x7fadcc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 0x7fadcc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadcc005210 0x7fadcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b2f50 con 0x7fadcc004850 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc00c0b0 con 0x7fadcc005210 2023-11-24T06:25:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc00c270 con 0x7fadcc003a00 2023-11-24T06:25:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.752+0000 7fade37fe700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 0x7fadcc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade3fff700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc003a00 0x7fadcc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade8c66700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadcc005210 0x7fadcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 -- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc003a00 msgr2=0x7fadcc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc003a00 0x7fadcc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 -- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadcc005210 msgr2=0x7fadcc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadcc005210 0x7fadcc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 -- 172.21.15.114:0/1317606023 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0035b0 con 0x7fadcc004850 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade8c66700 1 -- 172.21.15.114:0/1317606023 reap_dead start 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade3fff700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc003a00 0x7fadcc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade37fe700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 0x7fadcc004cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009b60 tx=0x7fadcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea5fd58eca38da8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800e6c0 con 0x7fadcc004850 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800e820 con 0x7fadcc004850 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fade2ffd700 1 -- 172.21.15.114:0/1317606023 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800eb90 con 0x7fadcc004850 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fadea6c9700 1 -- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 msgr2=0x7fadcc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.753+0000 7fadea6c9700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 0x7fadcc004cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009b60 tx=0x7fadcc005880 comp rx=0 tx=0).stop 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.754+0000 7fadea6c9700 1 -- 172.21.15.114:0/1317606023 shutdown_connections 2023-11-24T06:25:22.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.754+0000 7fadea6c9700 1 --2- 172.21.15.114:0/1317606023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadcc004850 0x7fadcc004cd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.754+0000 7fadea6c9700 1 -- 172.21.15.114:0/1317606023 >> 172.21.15.114:0/1317606023 conn(0x7fade40731a0 msgr2=0x7fade409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.754+0000 7fadea6c9700 1 -- 172.21.15.114:0/1317606023 shutdown_connections 2023-11-24T06:25:22.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.754+0000 7fadea6c9700 1 -- 172.21.15.114:0/1317606023 wait complete. 2023-11-24T06:25:22.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.755+0000 7fadea6c9700 1 Processor -- start 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.755+0000 7fadea6c9700 1 -- start start 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc004850 0x7fade40b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade40b5910 0x7fade40ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b6bf0 con 0x7fade4157240 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b6ed0 con 0x7fade40b5910 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b71b0 con 0x7fadcc004850 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39886/0 (socket says 172.21.15.114:39886) 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 -- 172.21.15.114:0/575988187 learned_addr learned my addr 172.21.15.114:0/575988187 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade3fff700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc004850 0x7fade40b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade8c66700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade40b5910 0x7fade40ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 -- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc004850 msgr2=0x7fade40b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc004850 0x7fade40b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 -- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade40b5910 msgr2=0x7fade40ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade40b5910 0x7fade40ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 -- 172.21.15.114:0/575988187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40b7490 con 0x7fade4157240 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade37fe700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c080 tx=0x7fadd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800e6c0 con 0x7fade4157240 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8003ad0 con 0x7fade4157240 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40b7770 con 0x7fade4157240 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003e80 con 0x7fade4157240 2023-11-24T06:25:22.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.756+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade406aff0 con 0x7fade4157240 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.757+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade4067f60 con 0x7fade4157240 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.758+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadd8013470 con 0x7fade4157240 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.758+0000 7fade0ff9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 0x7fadd006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.758+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd808aee0 con 0x7fade4157240 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.758+0000 7fade3fff700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 0x7fadd006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.759+0000 7fade3fff700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 0x7fadd006d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001200 tx=0x7fadd4007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:22.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.763+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadd8056140 con 0x7fade4157240 2023-11-24T06:25:22.949 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.946+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fade41604e0 con 0x7fadd006aed0 2023-11-24T06:25:22.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.954+0000 7fade0ff9700 1 -- 172.21.15.114:0/575988187 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288819 (secure 0 0 0) 0x7fadd4004920 con 0x7fadd006aed0 2023-11-24T06:25:22.961 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:22.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 msgr2=0x7fadd006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 0x7fadd006d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001200 tx=0x7fadd4007040 comp rx=0 tx=0).stop 2023-11-24T06:25:22.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 msgr2=0x7fade40cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c080 tx=0x7fadd80047c0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 shutdown_connections 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fadd006aed0 0x7fadd006d390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade40b5910 0x7fade40ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadcc004850 0x7fade40b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 --2- 172.21.15.114:0/575988187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fade4157240 0x7fade40cdf10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.957+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 >> 172.21.15.114:0/575988187 conn(0x7fade40731a0 msgr2=0x7fade415f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.958+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 shutdown_connections 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:22.958+0000 7fadea6c9700 1 -- 172.21.15.114:0/575988187 wait complete. 2023-11-24T06:25:22.965 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2023-11-24T06:25:21.719996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220356,"kb_used_data":4160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754364,"statfs":{"total":773094113280,"available":772868468736,"internally_reserved":0,"allocated":4259840,"data_stored":2357796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.256041"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500950+0000","last_change":"2023-11-24T06:25:18.502262+0000","last_active":"2023-11-24T06:25:19.500950+0000","last_peered":"2023-11-24T06:25:19.500950+0000","last_clean":"2023-11-24T06:25:19.500950+0000","last_became_active":"2023-11-24T06:25:18.501902+0000","last_became_peered":"2023-11-24T06:25:18.501902+0000","last_unstale":"2023-11-24T06:25:19.500950+0000","last_undegraded":"2023-11-24T06:25:19.500950+0000","last_fullsized":"2023-11-24T06:25:19.500950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:04.180393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499506+0000","last_change":"2023-11-24T06:25:18.514731+0000","last_active":"2023-11-24T06:25:19.499506+0000","last_peered":"2023-11-24T06:25:19.499506+0000","last_clean":"2023-11-24T06:25:19.499506+0000","last_became_active":"2023-11-24T06:25:18.514109+0000","last_became_peered":"2023-11-24T06:25:18.514109+0000","last_unstale":"2023-11-24T06:25:19.499506+0000","last_undegraded":"2023-11-24T06:25:19.499506+0000","last_fullsized":"2023-11-24T06:25:19.499506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:40.775365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511733+0000","last_change":"2023-11-24T06:25:18.509947+0000","last_active":"2023-11-24T06:25:18.511733+0000","last_peered":"2023-11-24T06:25:18.511733+0000","last_clean":"2023-11-24T06:25:18.511733+0000","last_became_active":"2023-11-24T06:25:18.509264+0000","last_became_peered":"2023-11-24T06:25:18.509264+0000","last_unstale":"2023-11-24T06:25:18.511733+0000","last_undegraded":"2023-11-24T06:25:18.511733+0000","last_fullsized":"2023-11-24T06:25:18.511733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:33.645797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639033+0000","last_change":"2023-11-24T06:25:18.501205+0000","last_active":"2023-11-24T06:25:19.639033+0000","last_peered":"2023-11-24T06:25:19.639033+0000","last_clean":"2023-11-24T06:25:19.639033+0000","last_became_active":"2023-11-24T06:25:18.500554+0000","last_became_peered":"2023-11-24T06:25:18.500554+0000","last_unstale":"2023-11-24T06:25:19.639033+0000","last_undegraded":"2023-11-24T06:25:19.639033+0000","last_fullsized":"2023-11-24T06:25:19.639033+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:17.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.516907+0000","last_change":"2023-11-24T06:25:18.511577+0000","last_active":"2023-11-24T06:25:18.516907+0000","last_peered":"2023-11-24T06:25:18.516907+0000","last_clean":"2023-11-24T06:25:18.516907+0000","last_became_active":"2023-11-24T06:25:18.507913+0000","last_became_peered":"2023-11-24T06:25:18.507913+0000","last_unstale":"2023-11-24T06:25:18.516907+0000","last_undegraded":"2023-11-24T06:25:18.516907+0000","last_fullsized":"2023-11-24T06:25:18.516907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:19.554771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499391+0000","last_change":"2023-11-24T06:25:18.517349+0000","last_active":"2023-11-24T06:25:19.499391+0000","last_peered":"2023-11-24T06:25:19.499391+0000","last_clean":"2023-11-24T06:25:19.499391+0000","last_became_active":"2023-11-24T06:25:18.517141+0000","last_became_peered":"2023-11-24T06:25:18.517141+0000","last_unstale":"2023-11-24T06:25:19.499391+0000","last_undegraded":"2023-11-24T06:25:19.499391+0000","last_fullsized":"2023-11-24T06:25:19.499391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:24.205989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500588+0000","last_change":"2023-11-24T06:25:18.504344+0000","last_active":"2023-11-24T06:25:19.500588+0000","last_peered":"2023-11-24T06:25:19.500588+0000","last_clean":"2023-11-24T06:25:19.500588+0000","last_became_active":"2023-11-24T06:25:18.504087+0000","last_became_peered":"2023-11-24T06:25:18.504087+0000","last_unstale":"2023-11-24T06:25:19.500588+0000","last_undegraded":"2023-11-24T06:25:19.500588+0000","last_fullsized":"2023-11-24T06:25:19.500588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:11.289535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500287+0000","last_change":"2023-11-24T06:25:18.494796+0000","last_active":"2023-11-24T06:25:19.500287+0000","last_peered":"2023-11-24T06:25:19.500287+0000","last_clean":"2023-11-24T06:25:19.500287+0000","last_became_active":"2023-11-24T06:25:18.494514+0000","last_became_peered":"2023-11-24T06:25:18.494514+0000","last_unstale":"2023-11-24T06:25:19.500287+0000","last_undegraded":"2023-11-24T06:25:19.500287+0000","last_fullsized":"2023-11-24T06:25:19.500287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:08.580752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.497546+0000","last_change":"2023-11-24T06:25:18.494885+0000","last_active":"2023-11-24T06:25:18.497546+0000","last_peered":"2023-11-24T06:25:18.497546+0000","last_clean":"2023-11-24T06:25:18.497546+0000","last_became_active":"2023-11-24T06:25:18.494604+0000","last_became_peered":"2023-11-24T06:25:18.494604+0000","last_unstale":"2023-11-24T06:25:18.497546+0000","last_undegraded":"2023-11-24T06:25:18.497546+0000","last_fullsized":"2023-11-24T06:25:18.497546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:21.873198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639215+0000","last_change":"2023-11-24T06:25:18.518929+0000","last_active":"2023-11-24T06:25:19.639215+0000","last_peered":"2023-11-24T06:25:19.639215+0000","last_clean":"2023-11-24T06:25:19.639215+0000","last_became_active":"2023-11-24T06:25:18.518663+0000","last_became_peered":"2023-11-24T06:25:18.518663+0000","last_unstale":"2023-11-24T06:25:19.639215+0000","last_undegraded":"2023-11-24T06:25:19.639215+0000","last_fullsized":"2023-11-24T06:25:19.639215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:24.512975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500622+0000","last_change":"2023-11-24T06:25:18.495155+0000","last_active":"2023-11-24T06:25:19.500622+0000","last_peered":"2023-11-24T06:25:19.500622+0000","last_clean":"2023-11-24T06:25:19.500622+0000","last_became_active":"2023-11-24T06:25:18.494831+0000","last_became_peered":"2023-11-24T06:25:18.494831+0000","last_unstale":"2023-11-24T06:25:19.500622+0000","last_undegraded":"2023-11-24T06:25:19.500622+0000","last_fullsized":"2023-11-24T06:25:19.500622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:23.861385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506856+0000","last_change":"2023-11-24T06:25:18.500859+0000","last_active":"2023-11-24T06:25:18.506856+0000","last_peered":"2023-11-24T06:25:18.506856+0000","last_clean":"2023-11-24T06:25:18.506856+0000","last_became_active":"2023-11-24T06:25:18.500586+0000","last_became_peered":"2023-11-24T06:25:18.500586+0000","last_unstale":"2023-11-24T06:25:18.506856+0000","last_undegraded":"2023-11-24T06:25:18.506856+0000","last_fullsized":"2023-11-24T06:25:18.506856+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:14.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505326+0000","last_change":"2023-11-24T06:25:18.501559+0000","last_active":"2023-11-24T06:25:18.505326+0000","last_peered":"2023-11-24T06:25:18.505326+0000","last_clean":"2023-11-24T06:25:18.505326+0000","last_became_active":"2023-11-24T06:25:18.501275+0000","last_became_peered":"2023-11-24T06:25:18.501275+0000","last_unstale":"2023-11-24T06:25:18.505326+0000","last_undegraded":"2023-11-24T06:25:18.505326+0000","last_fullsized":"2023-11-24T06:25:18.505326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:20.171874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499403+0000","last_change":"2023-11-24T06:25:18.512608+0000","last_active":"2023-11-24T06:25:19.499403+0000","last_peered":"2023-11-24T06:25:19.499403+0000","last_clean":"2023-11-24T06:25:19.499403+0000","last_became_active":"2023-11-24T06:25:18.507454+0000","last_became_peered":"2023-11-24T06:25:18.507454+0000","last_unstale":"2023-11-24T06:25:19.499403+0000","last_undegraded":"2023-11-24T06:25:19.499403+0000","last_fullsized":"2023-11-24T06:25:19.499403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.723713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507172+0000","last_change":"2023-11-24T06:25:18.505091+0000","last_active":"2023-11-24T06:25:18.507172+0000","last_peered":"2023-11-24T06:25:18.507172+0000","last_clean":"2023-11-24T06:25:18.507172+0000","last_became_active":"2023-11-24T06:25:18.504803+0000","last_became_peered":"2023-11-24T06:25:18.504803+0000","last_unstale":"2023-11-24T06:25:18.507172+0000","last_undegraded":"2023-11-24T06:25:18.507172+0000","last_fullsized":"2023-11-24T06:25:18.507172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:15.336037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500857+0000","last_change":"2023-11-24T06:25:18.500526+0000","last_active":"2023-11-24T06:25:19.500857+0000","last_peered":"2023-11-24T06:25:19.500857+0000","last_clean":"2023-11-24T06:25:19.500857+0000","last_became_active":"2023-11-24T06:25:18.500276+0000","last_became_peered":"2023-11-24T06:25:18.500276+0000","last_unstale":"2023-11-24T06:25:19.500857+0000","last_undegraded":"2023-11-24T06:25:19.500857+0000","last_fullsized":"2023-11-24T06:25:19.500857+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:11.332932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501335+0000","last_change":"2023-11-24T06:25:18.503037+0000","last_active":"2023-11-24T06:25:19.501335+0000","last_peered":"2023-11-24T06:25:19.501335+0000","last_clean":"2023-11-24T06:25:19.501335+0000","last_became_active":"2023-11-24T06:25:18.502777+0000","last_became_peered":"2023-11-24T06:25:18.502777+0000","last_unstale":"2023-11-24T06:25:19.501335+0000","last_undegraded":"2023-11-24T06:25:19.501335+0000","last_fullsized":"2023-11-24T06:25:19.501335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:40.336914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499609+0000","last_change":"2023-11-24T06:25:18.515651+0000","last_active":"2023-11-24T06:25:19.499609+0000","last_peered":"2023-11-24T06:25:19.499609+0000","last_clean":"2023-11-24T06:25:19.499609+0000","last_became_active":"2023-11-24T06:25:18.514154+0000","last_became_peered":"2023-11-24T06:25:18.514154+0000","last_unstale":"2023-11-24T06:25:19.499609+0000","last_undegraded":"2023-11-24T06:25:19.499609+0000","last_fullsized":"2023-11-24T06:25:19.499609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:12.018354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499162+0000","last_change":"2023-11-24T06:25:18.516522+0000","last_active":"2023-11-24T06:25:19.499162+0000","last_peered":"2023-11-24T06:25:19.499162+0000","last_clean":"2023-11-24T06:25:19.499162+0000","last_became_active":"2023-11-24T06:25:18.514196+0000","last_became_peered":"2023-11-24T06:25:18.514196+0000","last_unstale":"2023-11-24T06:25:19.499162+0000","last_undegraded":"2023-11-24T06:25:19.499162+0000","last_fullsized":"2023-11-24T06:25:19.499162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:01.050108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505631+0000","last_change":"2023-11-24T06:25:18.499047+0000","last_active":"2023-11-24T06:25:18.505631+0000","last_peered":"2023-11-24T06:25:18.505631+0000","last_clean":"2023-11-24T06:25:18.505631+0000","last_became_active":"2023-11-24T06:25:18.498778+0000","last_became_peered":"2023-11-24T06:25:18.498778+0000","last_unstale":"2023-11-24T06:25:18.505631+0000","last_undegraded":"2023-11-24T06:25:18.505631+0000","last_fullsized":"2023-11-24T06:25:18.505631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:32.206913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507523+0000","last_change":"2023-11-24T06:25:18.500860+0000","last_active":"2023-11-24T06:25:18.507523+0000","last_peered":"2023-11-24T06:25:18.507523+0000","last_clean":"2023-11-24T06:25:18.507523+0000","last_became_active":"2023-11-24T06:25:18.500583+0000","last_became_peered":"2023-11-24T06:25:18.500583+0000","last_unstale":"2023-11-24T06:25:18.507523+0000","last_undegraded":"2023-11-24T06:25:18.507523+0000","last_fullsized":"2023-11-24T06:25:18.507523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:16.853872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507225+0000","last_change":"2023-11-24T06:25:18.502994+0000","last_active":"2023-11-24T06:25:18.507225+0000","last_peered":"2023-11-24T06:25:18.507225+0000","last_clean":"2023-11-24T06:25:18.507225+0000","last_became_active":"2023-11-24T06:25:18.502697+0000","last_became_peered":"2023-11-24T06:25:18.502697+0000","last_unstale":"2023-11-24T06:25:18.507225+0000","last_undegraded":"2023-11-24T06:25:18.507225+0000","last_fullsized":"2023-11-24T06:25:18.507225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:10.813108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688733+0000","last_change":"2023-11-24T06:25:18.502958+0000","last_active":"2023-11-24T06:25:19.688733+0000","last_peered":"2023-11-24T06:25:19.688733+0000","last_clean":"2023-11-24T06:25:19.688733+0000","last_became_active":"2023-11-24T06:25:18.502635+0000","last_became_peered":"2023-11-24T06:25:18.502635+0000","last_unstale":"2023-11-24T06:25:19.688733+0000","last_undegraded":"2023-11-24T06:25:19.688733+0000","last_fullsized":"2023-11-24T06:25:19.688733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:45.204085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.508457+0000","last_change":"2023-11-24T06:25:18.506042+0000","last_active":"2023-11-24T06:25:18.508457+0000","last_peered":"2023-11-24T06:25:18.508457+0000","last_clean":"2023-11-24T06:25:18.508457+0000","last_became_active":"2023-11-24T06:25:18.505781+0000","last_became_peered":"2023-11-24T06:25:18.505781+0000","last_unstale":"2023-11-24T06:25:18.508457+0000","last_undegraded":"2023-11-24T06:25:18.508457+0000","last_fullsized":"2023-11-24T06:25:18.508457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:45.204085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639307+0000","last_change":"2023-11-24T06:25:18.501979+0000","last_active":"2023-11-24T06:25:19.639307+0000","last_peered":"2023-11-24T06:25:19.639307+0000","last_clean":"2023-11-24T06:25:19.639307+0000","last_became_active":"2023-11-24T06:25:18.500876+0000","last_became_peered":"2023-11-24T06:25:18.500876+0000","last_unstale":"2023-11-24T06:25:19.639307+0000","last_undegraded":"2023-11-24T06:25:19.639307+0000","last_fullsized":"2023-11-24T06:25:19.639307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:23.861385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499632+0000","last_change":"2023-11-24T06:25:18.515922+0000","last_active":"2023-11-24T06:25:19.499632+0000","last_peered":"2023-11-24T06:25:19.499632+0000","last_clean":"2023-11-24T06:25:19.499632+0000","last_became_active":"2023-11-24T06:25:18.514414+0000","last_became_peered":"2023-11-24T06:25:18.514414+0000","last_unstale":"2023-11-24T06:25:19.499632+0000","last_undegraded":"2023-11-24T06:25:19.499632+0000","last_fullsized":"2023-11-24T06:25:19.499632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:19.331758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639217+0000","last_change":"2023-11-24T06:25:18.509473+0000","last_active":"2023-11-24T06:25:19.639217+0000","last_peered":"2023-11-24T06:25:19.639217+0000","last_clean":"2023-11-24T06:25:19.639217+0000","last_became_active":"2023-11-24T06:25:18.508623+0000","last_became_peered":"2023-11-24T06:25:18.508623+0000","last_unstale":"2023-11-24T06:25:19.639217+0000","last_undegraded":"2023-11-24T06:25:19.639217+0000","last_fullsized":"2023-11-24T06:25:19.639217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:51.733991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506873+0000","last_change":"2023-11-24T06:25:18.504009+0000","last_active":"2023-11-24T06:25:18.506873+0000","last_peered":"2023-11-24T06:25:18.506873+0000","last_clean":"2023-11-24T06:25:18.506873+0000","last_became_active":"2023-11-24T06:25:18.503112+0000","last_became_peered":"2023-11-24T06:25:18.503112+0000","last_unstale":"2023-11-24T06:25:18.506873+0000","last_undegraded":"2023-11-24T06:25:18.506873+0000","last_fullsized":"2023-11-24T06:25:18.506873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:04.180393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510045+0000","last_change":"2023-11-24T06:25:18.508249+0000","last_active":"2023-11-24T06:25:18.510045+0000","last_peered":"2023-11-24T06:25:18.510045+0000","last_clean":"2023-11-24T06:25:18.510045+0000","last_became_active":"2023-11-24T06:25:18.507964+0000","last_became_peered":"2023-11-24T06:25:18.507964+0000","last_unstale":"2023-11-24T06:25:18.510045+0000","last_undegraded":"2023-11-24T06:25:18.510045+0000","last_fullsized":"2023-11-24T06:25:18.510045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:18.757773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500415+0000","last_change":"2023-11-24T06:25:18.502562+0000","last_active":"2023-11-24T06:25:19.500415+0000","last_peered":"2023-11-24T06:25:19.500415+0000","last_clean":"2023-11-24T06:25:19.500415+0000","last_became_active":"2023-11-24T06:25:18.501650+0000","last_became_peered":"2023-11-24T06:25:18.501650+0000","last_unstale":"2023-11-24T06:25:19.500415+0000","last_undegraded":"2023-11-24T06:25:19.500415+0000","last_fullsized":"2023-11-24T06:25:19.500415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:24.512975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500632+0000","last_change":"2023-11-24T06:25:18.509286+0000","last_active":"2023-11-24T06:25:19.500632+0000","last_peered":"2023-11-24T06:25:19.500632+0000","last_clean":"2023-11-24T06:25:19.500632+0000","last_became_active":"2023-11-24T06:25:18.508803+0000","last_became_peered":"2023-11-24T06:25:18.508803+0000","last_unstale":"2023-11-24T06:25:19.500632+0000","last_undegraded":"2023-11-24T06:25:19.500632+0000","last_fullsized":"2023-11-24T06:25:19.500632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:26:17.506990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501204+0000","last_change":"2023-11-24T06:25:18.499234+0000","last_active":"2023-11-24T06:25:19.501204+0000","last_peered":"2023-11-24T06:25:19.501204+0000","last_clean":"2023-11-24T06:25:19.501204+0000","last_became_active":"2023-11-24T06:25:18.498958+0000","last_became_peered":"2023-11-24T06:25:18.498958+0000","last_unstale":"2023-11-24T06:25:19.501204+0000","last_undegraded":"2023-11-24T06:25:19.501204+0000","last_fullsized":"2023-11-24T06:25:19.501204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.723713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688839+0000","last_change":"2023-11-24T06:25:18.503692+0000","last_active":"2023-11-24T06:25:19.688839+0000","last_peered":"2023-11-24T06:25:19.688839+0000","last_clean":"2023-11-24T06:25:19.688839+0000","last_became_active":"2023-11-24T06:25:18.503381+0000","last_became_peered":"2023-11-24T06:25:18.503381+0000","last_unstale":"2023-11-24T06:25:19.688839+0000","last_undegraded":"2023-11-24T06:25:19.688839+0000","last_fullsized":"2023-11-24T06:25:19.688839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:14.354831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501171+0000","last_change":"2023-11-24T06:25:17.488220+0000","last_active":"2023-11-24T06:25:19.501171+0000","last_peered":"2023-11-24T06:25:19.501171+0000","last_clean":"2023-11-24T06:25:19.501171+0000","last_became_active":"2023-11-24T06:25:17.487826+0000","last_became_peered":"2023-11-24T06:25:17.487826+0000","last_unstale":"2023-11-24T06:25:19.501171+0000","last_undegraded":"2023-11-24T06:25:19.501171+0000","last_fullsized":"2023-11-24T06:25:19.501171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:25.824047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501109+0000","last_change":"2023-11-24T06:25:18.501338+0000","last_active":"2023-11-24T06:25:19.501109+0000","last_peered":"2023-11-24T06:25:19.501109+0000","last_clean":"2023-11-24T06:25:19.501109+0000","last_became_active":"2023-11-24T06:25:18.501008+0000","last_became_peered":"2023-11-24T06:25:18.501008+0000","last_unstale":"2023-11-24T06:25:19.501109+0000","last_undegraded":"2023-11-24T06:25:19.501109+0000","last_fullsized":"2023-11-24T06:25:19.501109+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:08.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688890+0000","last_change":"2023-11-24T06:25:17.486512+0000","last_active":"2023-11-24T06:25:19.688890+0000","last_peered":"2023-11-24T06:25:19.688890+0000","last_clean":"2023-11-24T06:25:19.688890+0000","last_became_active":"2023-11-24T06:25:17.486091+0000","last_became_peered":"2023-11-24T06:25:17.486091+0000","last_unstale":"2023-11-24T06:25:19.688890+0000","last_undegraded":"2023-11-24T06:25:19.688890+0000","last_fullsized":"2023-11-24T06:25:19.688890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:25.408123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510350+0000","last_change":"2023-11-24T06:25:18.508205+0000","last_active":"2023-11-24T06:25:18.510350+0000","last_peered":"2023-11-24T06:25:18.510350+0000","last_clean":"2023-11-24T06:25:18.510350+0000","last_became_active":"2023-11-24T06:25:18.507911+0000","last_became_peered":"2023-11-24T06:25:18.507911+0000","last_unstale":"2023-11-24T06:25:18.510350+0000","last_undegraded":"2023-11-24T06:25:18.510350+0000","last_fullsized":"2023-11-24T06:25:18.510350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:26:17.506990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499162+0000","last_change":"2023-11-24T06:25:17.485162+0000","last_active":"2023-11-24T06:25:19.499162+0000","last_peered":"2023-11-24T06:25:19.499162+0000","last_clean":"2023-11-24T06:25:19.499162+0000","last_became_active":"2023-11-24T06:25:17.484879+0000","last_became_peered":"2023-11-24T06:25:17.484879+0000","last_unstale":"2023-11-24T06:25:19.499162+0000","last_undegraded":"2023-11-24T06:25:19.499162+0000","last_fullsized":"2023-11-24T06:25:19.499162+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:22.851277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639294+0000","last_change":"2023-11-24T06:25:18.508950+0000","last_active":"2023-11-24T06:25:19.639294+0000","last_peered":"2023-11-24T06:25:19.639294+0000","last_clean":"2023-11-24T06:25:19.639294+0000","last_became_active":"2023-11-24T06:25:18.508429+0000","last_became_peered":"2023-11-24T06:25:18.508429+0000","last_unstale":"2023-11-24T06:25:19.639294+0000","last_undegraded":"2023-11-24T06:25:19.639294+0000","last_fullsized":"2023-11-24T06:25:19.639294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:11.289535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.514735+0000","last_change":"2023-11-24T06:25:17.486491+0000","last_active":"2023-11-24T06:25:18.514735+0000","last_peered":"2023-11-24T06:25:18.514735+0000","last_clean":"2023-11-24T06:25:18.514735+0000","last_became_active":"2023-11-24T06:25:17.486232+0000","last_became_peered":"2023-11-24T06:25:17.486232+0000","last_unstale":"2023-11-24T06:25:18.514735+0000","last_undegraded":"2023-11-24T06:25:18.514735+0000","last_fullsized":"2023-11-24T06:25:18.514735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:51.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500565+0000","last_change":"2023-11-24T06:25:18.496438+0000","last_active":"2023-11-24T06:25:19.500565+0000","last_peered":"2023-11-24T06:25:19.500565+0000","last_clean":"2023-11-24T06:25:19.500565+0000","last_became_active":"2023-11-24T06:25:18.496150+0000","last_became_peered":"2023-11-24T06:25:18.496150+0000","last_unstale":"2023-11-24T06:25:19.500565+0000","last_undegraded":"2023-11-24T06:25:19.500565+0000","last_fullsized":"2023-11-24T06:25:19.500565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:51.733991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.515927+0000","last_change":"2023-11-24T06:25:17.481913+0000","last_active":"2023-11-24T06:25:18.515927+0000","last_peered":"2023-11-24T06:25:18.515927+0000","last_clean":"2023-11-24T06:25:18.515927+0000","last_became_active":"2023-11-24T06:25:17.481634+0000","last_became_peered":"2023-11-24T06:25:17.481634+0000","last_unstale":"2023-11-24T06:25:18.515927+0000","last_undegraded":"2023-11-24T06:25:18.515927+0000","last_fullsized":"2023-11-24T06:25:18.515927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:34.526611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506854+0000","last_change":"2023-11-24T06:25:18.503501+0000","last_active":"2023-11-24T06:25:18.506854+0000","last_peered":"2023-11-24T06:25:18.506854+0000","last_clean":"2023-11-24T06:25:18.506854+0000","last_became_active":"2023-11-24T06:25:18.503030+0000","last_became_peered":"2023-11-24T06:25:18.503030+0000","last_unstale":"2023-11-24T06:25:18.506854+0000","last_undegraded":"2023-11-24T06:25:18.506854+0000","last_fullsized":"2023-11-24T06:25:18.506854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:08.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.518502+0000","last_change":"2023-11-24T06:25:17.482000+0000","last_active":"2023-11-24T06:25:18.518502+0000","last_peered":"2023-11-24T06:25:18.518502+0000","last_clean":"2023-11-24T06:25:18.518502+0000","last_became_active":"2023-11-24T06:25:17.481670+0000","last_became_peered":"2023-11-24T06:25:17.481670+0000","last_unstale":"2023-11-24T06:25:18.518502+0000","last_undegraded":"2023-11-24T06:25:18.518502+0000","last_fullsized":"2023-11-24T06:25:18.518502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:51.768193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.506750+0000","last_change":"2023-11-24T06:25:18.500238+0000","last_active":"2023-11-24T06:25:18.506750+0000","last_peered":"2023-11-24T06:25:18.506750+0000","last_clean":"2023-11-24T06:25:18.506750+0000","last_became_active":"2023-11-24T06:25:18.499798+0000","last_became_peered":"2023-11-24T06:25:18.499798+0000","last_unstale":"2023-11-24T06:25:18.506750+0000","last_undegraded":"2023-11-24T06:25:18.506750+0000","last_fullsized":"2023-11-24T06:25:18.506750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:12.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498976+0000","last_change":"2023-11-24T06:25:17.481893+0000","last_active":"2023-11-24T06:25:19.498976+0000","last_peered":"2023-11-24T06:25:19.498976+0000","last_clean":"2023-11-24T06:25:19.498976+0000","last_became_active":"2023-11-24T06:25:17.481560+0000","last_became_peered":"2023-11-24T06:25:17.481560+0000","last_unstale":"2023-11-24T06:25:19.498976+0000","last_undegraded":"2023-11-24T06:25:19.498976+0000","last_fullsized":"2023-11-24T06:25:19.498976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:08.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638874+0000","last_change":"2023-11-24T06:25:18.497901+0000","last_active":"2023-11-24T06:25:19.638874+0000","last_peered":"2023-11-24T06:25:19.638874+0000","last_clean":"2023-11-24T06:25:19.638874+0000","last_became_active":"2023-11-24T06:25:18.497630+0000","last_became_peered":"2023-11-24T06:25:18.497630+0000","last_unstale":"2023-11-24T06:25:19.638874+0000","last_undegraded":"2023-11-24T06:25:19.638874+0000","last_fullsized":"2023-11-24T06:25:19.638874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:33.645797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.486168+0000","last_change":"2023-11-24T06:25:17.483963+0000","last_active":"2023-11-24T06:25:18.486168+0000","last_peered":"2023-11-24T06:25:18.486168+0000","last_clean":"2023-11-24T06:25:18.486168+0000","last_became_active":"2023-11-24T06:25:17.483566+0000","last_became_peered":"2023-11-24T06:25:17.483566+0000","last_unstale":"2023-11-24T06:25:18.486168+0000","last_undegraded":"2023-11-24T06:25:18.486168+0000","last_fullsized":"2023-11-24T06:25:18.486168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:03.924569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.517476+0000","last_change":"2023-11-24T06:25:18.516389+0000","last_active":"2023-11-24T06:25:18.517476+0000","last_peered":"2023-11-24T06:25:18.517476+0000","last_clean":"2023-11-24T06:25:18.517476+0000","last_became_active":"2023-11-24T06:25:18.516109+0000","last_became_peered":"2023-11-24T06:25:18.516109+0000","last_unstale":"2023-11-24T06:25:18.517476+0000","last_undegraded":"2023-11-24T06:25:18.517476+0000","last_fullsized":"2023-11-24T06:25:18.517476+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:08.943735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499423+0000","last_change":"2023-11-24T06:25:17.486575+0000","last_active":"2023-11-24T06:25:19.499423+0000","last_peered":"2023-11-24T06:25:19.499423+0000","last_clean":"2023-11-24T06:25:19.499423+0000","last_became_active":"2023-11-24T06:25:17.486301+0000","last_became_peered":"2023-11-24T06:25:17.486301+0000","last_unstale":"2023-11-24T06:25:19.499423+0000","last_undegraded":"2023-11-24T06:25:19.499423+0000","last_fullsized":"2023-11-24T06:25:19.499423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:50.723882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499383+0000","last_change":"2023-11-24T06:25:18.516055+0000","last_active":"2023-11-24T06:25:19.499383+0000","last_peered":"2023-11-24T06:25:19.499383+0000","last_clean":"2023-11-24T06:25:19.499383+0000","last_became_active":"2023-11-24T06:25:18.515789+0000","last_became_peered":"2023-11-24T06:25:18.515789+0000","last_unstale":"2023-11-24T06:25:19.499383+0000","last_undegraded":"2023-11-24T06:25:19.499383+0000","last_fullsized":"2023-11-24T06:25:19.499383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:05.460194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688796+0000","last_change":"2023-11-24T06:25:17.482356+0000","last_active":"2023-11-24T06:25:19.688796+0000","last_peered":"2023-11-24T06:25:19.688796+0000","last_clean":"2023-11-24T06:25:19.688796+0000","last_became_active":"2023-11-24T06:25:17.482087+0000","last_became_peered":"2023-11-24T06:25:17.482087+0000","last_unstale":"2023-11-24T06:25:19.688796+0000","last_undegraded":"2023-11-24T06:25:19.688796+0000","last_fullsized":"2023-11-24T06:25:19.688796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:11.578791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499231+0000","last_change":"2023-11-24T06:25:18.514073+0000","last_active":"2023-11-24T06:25:19.499231+0000","last_peered":"2023-11-24T06:25:19.499231+0000","last_clean":"2023-11-24T06:25:19.499231+0000","last_became_active":"2023-11-24T06:25:18.513216+0000","last_became_peered":"2023-11-24T06:25:18.513216+0000","last_unstale":"2023-11-24T06:25:19.499231+0000","last_undegraded":"2023-11-24T06:25:19.499231+0000","last_fullsized":"2023-11-24T06:25:19.499231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:11.332932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.510567+0000","last_change":"2023-11-24T06:25:17.477898+0000","last_active":"2023-11-24T06:25:18.510567+0000","last_peered":"2023-11-24T06:25:18.510567+0000","last_clean":"2023-11-24T06:25:18.510567+0000","last_became_active":"2023-11-24T06:25:17.477479+0000","last_became_peered":"2023-11-24T06:25:17.477479+0000","last_unstale":"2023-11-24T06:25:18.510567+0000","last_undegraded":"2023-11-24T06:25:18.510567+0000","last_fullsized":"2023-11-24T06:25:18.510567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:16.417587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511752+0000","last_change":"2023-11-24T06:25:18.510150+0000","last_active":"2023-11-24T06:25:18.511752+0000","last_peered":"2023-11-24T06:25:18.511752+0000","last_clean":"2023-11-24T06:25:18.511752+0000","last_became_active":"2023-11-24T06:25:18.509661+0000","last_became_peered":"2023-11-24T06:25:18.509661+0000","last_unstale":"2023-11-24T06:25:18.511752+0000","last_undegraded":"2023-11-24T06:25:18.511752+0000","last_fullsized":"2023-11-24T06:25:18.511752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:35.536720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499320+0000","last_change":"2023-11-24T06:25:17.486745+0000","last_active":"2023-11-24T06:25:19.499320+0000","last_peered":"2023-11-24T06:25:19.499320+0000","last_clean":"2023-11-24T06:25:19.499320+0000","last_became_active":"2023-11-24T06:25:17.486318+0000","last_became_peered":"2023-11-24T06:25:17.486318+0000","last_unstale":"2023-11-24T06:25:19.499320+0000","last_undegraded":"2023-11-24T06:25:19.499320+0000","last_fullsized":"2023-11-24T06:25:19.499320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.279426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638893+0000","last_change":"2023-11-24T06:25:18.497371+0000","last_active":"2023-11-24T06:25:19.638893+0000","last_peered":"2023-11-24T06:25:19.638893+0000","last_clean":"2023-11-24T06:25:19.638893+0000","last_became_active":"2023-11-24T06:25:18.497144+0000","last_became_peered":"2023-11-24T06:25:18.497144+0000","last_unstale":"2023-11-24T06:25:19.638893+0000","last_undegraded":"2023-11-24T06:25:19.638893+0000","last_fullsized":"2023-11-24T06:25:19.638893+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:09.769193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511835+0000","last_change":"2023-11-24T06:25:17.483866+0000","last_active":"2023-11-24T06:25:18.511835+0000","last_peered":"2023-11-24T06:25:18.511835+0000","last_clean":"2023-11-24T06:25:18.511835+0000","last_became_active":"2023-11-24T06:25:17.483588+0000","last_became_peered":"2023-11-24T06:25:17.483588+0000","last_unstale":"2023-11-24T06:25:18.511835+0000","last_undegraded":"2023-11-24T06:25:18.511835+0000","last_fullsized":"2023-11-24T06:25:18.511835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:03.170284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499337+0000","last_change":"2023-11-24T06:25:18.516869+0000","last_active":"2023-11-24T06:25:19.499337+0000","last_peered":"2023-11-24T06:25:19.499337+0000","last_clean":"2023-11-24T06:25:19.499337+0000","last_became_active":"2023-11-24T06:25:18.513211+0000","last_became_peered":"2023-11-24T06:25:18.513211+0000","last_unstale":"2023-11-24T06:25:19.499337+0000","last_undegraded":"2023-11-24T06:25:19.499337+0000","last_fullsized":"2023-11-24T06:25:19.499337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:40.336914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638986+0000","last_change":"2023-11-24T06:25:17.487172+0000","last_active":"2023-11-24T06:25:19.638986+0000","last_peered":"2023-11-24T06:25:19.638986+0000","last_clean":"2023-11-24T06:25:19.638986+0000","last_became_active":"2023-11-24T06:25:17.486795+0000","last_became_peered":"2023-11-24T06:25:17.486795+0000","last_unstale":"2023-11-24T06:25:19.638986+0000","last_undegraded":"2023-11-24T06:25:19.638986+0000","last_fullsized":"2023-11-24T06:25:19.638986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:29.274409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501079+0000","last_change":"2023-11-24T06:25:18.499346+0000","last_active":"2023-11-24T06:25:19.501079+0000","last_peered":"2023-11-24T06:25:19.501079+0000","last_clean":"2023-11-24T06:25:19.501079+0000","last_became_active":"2023-11-24T06:25:18.499062+0000","last_became_peered":"2023-11-24T06:25:18.499062+0000","last_unstale":"2023-11-24T06:25:19.501079+0000","last_undegraded":"2023-11-24T06:25:19.501079+0000","last_fullsized":"2023-11-24T06:25:19.501079+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:05.460194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499006+0000","last_change":"2023-11-24T06:25:17.485252+0000","last_active":"2023-11-24T06:25:19.499006+0000","last_peered":"2023-11-24T06:25:19.499006+0000","last_clean":"2023-11-24T06:25:19.499006+0000","last_became_active":"2023-11-24T06:25:17.484972+0000","last_became_peered":"2023-11-24T06:25:17.484972+0000","last_unstale":"2023-11-24T06:25:19.499006+0000","last_undegraded":"2023-11-24T06:25:19.499006+0000","last_fullsized":"2023-11-24T06:25:19.499006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:16.417587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500479+0000","last_change":"2023-11-24T06:25:18.501889+0000","last_active":"2023-11-24T06:25:19.500479+0000","last_peered":"2023-11-24T06:25:19.500479+0000","last_clean":"2023-11-24T06:25:19.500479+0000","last_became_active":"2023-11-24T06:25:18.501582+0000","last_became_peered":"2023-11-24T06:25:18.501582+0000","last_unstale":"2023-11-24T06:25:19.500479+0000","last_undegraded":"2023-11-24T06:25:19.500479+0000","last_fullsized":"2023-11-24T06:25:19.500479+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:24.490953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.499062+0000","last_change":"2023-11-24T06:25:17.485031+0000","last_active":"2023-11-24T06:25:19.499062+0000","last_peered":"2023-11-24T06:25:19.499062+0000","last_clean":"2023-11-24T06:25:19.499062+0000","last_became_active":"2023-11-24T06:25:17.484743+0000","last_became_peered":"2023-11-24T06:25:17.484743+0000","last_unstale":"2023-11-24T06:25:19.499062+0000","last_undegraded":"2023-11-24T06:25:19.499062+0000","last_fullsized":"2023-11-24T06:25:19.499062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:07.570644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.499940+0000","last_change":"2023-11-24T06:25:18.496924+0000","last_active":"2023-11-24T06:25:18.499940+0000","last_peered":"2023-11-24T06:25:18.499940+0000","last_clean":"2023-11-24T06:25:18.499940+0000","last_became_active":"2023-11-24T06:25:18.496649+0000","last_became_peered":"2023-11-24T06:25:18.496649+0000","last_unstale":"2023-11-24T06:25:18.499940+0000","last_undegraded":"2023-11-24T06:25:18.499940+0000","last_fullsized":"2023-11-24T06:25:18.499940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:05:01.332186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639364+0000","last_change":"2023-11-24T06:25:17.482303+0000","last_active":"2023-11-24T06:25:19.639364+0000","last_peered":"2023-11-24T06:25:19.639364+0000","last_clean":"2023-11-24T06:25:19.639364+0000","last_became_active":"2023-11-24T06:25:17.482023+0000","last_became_peered":"2023-11-24T06:25:17.482023+0000","last_unstale":"2023-11-24T06:25:19.639364+0000","last_undegraded":"2023-11-24T06:25:19.639364+0000","last_fullsized":"2023-11-24T06:25:19.639364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:39.041951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505999+0000","last_change":"2023-11-24T06:25:18.498456+0000","last_active":"2023-11-24T06:25:18.505999+0000","last_peered":"2023-11-24T06:25:18.505999+0000","last_clean":"2023-11-24T06:25:18.505999+0000","last_became_active":"2023-11-24T06:25:18.498100+0000","last_became_peered":"2023-11-24T06:25:18.498100+0000","last_unstale":"2023-11-24T06:25:18.505999+0000","last_undegraded":"2023-11-24T06:25:18.505999+0000","last_fullsized":"2023-11-24T06:25:18.505999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:37:40.052060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511607+0000","last_change":"2023-11-24T06:25:17.486832+0000","last_active":"2023-11-24T06:25:18.511607+0000","last_peered":"2023-11-24T06:25:18.511607+0000","last_clean":"2023-11-24T06:25:18.511607+0000","last_became_active":"2023-11-24T06:25:17.486484+0000","last_became_peered":"2023-11-24T06:25:17.486484+0000","last_unstale":"2023-11-24T06:25:18.511607+0000","last_undegraded":"2023-11-24T06:25:18.511607+0000","last_fullsized":"2023-11-24T06:25:18.511607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:11.578791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639102+0000","last_change":"2023-11-24T06:25:18.503624+0000","last_active":"2023-11-24T06:25:19.639102+0000","last_peered":"2023-11-24T06:25:19.639102+0000","last_clean":"2023-11-24T06:25:19.639102+0000","last_became_active":"2023-11-24T06:25:18.500947+0000","last_became_peered":"2023-11-24T06:25:18.500947+0000","last_unstale":"2023-11-24T06:25:19.639102+0000","last_undegraded":"2023-11-24T06:25:19.639102+0000","last_fullsized":"2023-11-24T06:25:19.639102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:08.580752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.521788+0000","last_change":"2023-11-24T06:25:17.481879+0000","last_active":"2023-11-24T06:25:18.521788+0000","last_peered":"2023-11-24T06:25:18.521788+0000","last_clean":"2023-11-24T06:25:18.521788+0000","last_became_active":"2023-11-24T06:25:17.481562+0000","last_became_peered":"2023-11-24T06:25:17.481562+0000","last_unstale":"2023-11-24T06:25:18.521788+0000","last_undegraded":"2023-11-24T06:25:18.521788+0000","last_fullsized":"2023-11-24T06:25:18.521788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:18:06.213575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505414+0000","last_change":"2023-11-24T06:25:18.498968+0000","last_active":"2023-11-24T06:25:18.505414+0000","last_peered":"2023-11-24T06:25:18.505414+0000","last_clean":"2023-11-24T06:25:18.505414+0000","last_became_active":"2023-11-24T06:25:18.498682+0000","last_became_peered":"2023-11-24T06:25:18.498682+0000","last_unstale":"2023-11-24T06:25:18.505414+0000","last_undegraded":"2023-11-24T06:25:18.505414+0000","last_fullsized":"2023-11-24T06:25:18.505414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:34:51.769406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488821+0000","last_change":"2023-11-24T06:25:17.484206+0000","last_active":"2023-11-24T06:25:18.488821+0000","last_peered":"2023-11-24T06:25:18.488821+0000","last_clean":"2023-11-24T06:25:18.488821+0000","last_became_active":"2023-11-24T06:25:17.483937+0000","last_became_peered":"2023-11-24T06:25:17.483937+0000","last_unstale":"2023-11-24T06:25:18.488821+0000","last_undegraded":"2023-11-24T06:25:18.488821+0000","last_fullsized":"2023-11-24T06:25:18.488821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:07.933626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505253+0000","last_change":"2023-11-24T06:25:18.494824+0000","last_active":"2023-11-24T06:25:18.505253+0000","last_peered":"2023-11-24T06:25:18.505253+0000","last_clean":"2023-11-24T06:25:18.505253+0000","last_became_active":"2023-11-24T06:25:18.494537+0000","last_became_peered":"2023-11-24T06:25:18.494537+0000","last_unstale":"2023-11-24T06:25:18.505253+0000","last_undegraded":"2023-11-24T06:25:18.505253+0000","last_fullsized":"2023-11-24T06:25:18.505253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:09:16.811522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688911+0000","last_change":"2023-11-24T06:25:17.487142+0000","last_active":"2023-11-24T06:25:19.688911+0000","last_peered":"2023-11-24T06:25:19.688911+0000","last_clean":"2023-11-24T06:25:19.688911+0000","last_became_active":"2023-11-24T06:25:17.486871+0000","last_became_peered":"2023-11-24T06:25:17.486871+0000","last_unstale":"2023-11-24T06:25:19.688911+0000","last_undegraded":"2023-11-24T06:25:19.688911+0000","last_fullsized":"2023-11-24T06:25:19.688911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:34.526611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511673+0000","last_change":"2023-11-24T06:25:18.504881+0000","last_active":"2023-11-24T06:25:18.511673+0000","last_peered":"2023-11-24T06:25:18.511673+0000","last_clean":"2023-11-24T06:25:18.511673+0000","last_became_active":"2023-11-24T06:25:18.504611+0000","last_became_peered":"2023-11-24T06:25:18.504611+0000","last_unstale":"2023-11-24T06:25:18.511673+0000","last_undegraded":"2023-11-24T06:25:18.511673+0000","last_fullsized":"2023-11-24T06:25:18.511673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:47.180225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.501039+0000","last_change":"2023-11-24T06:25:17.486593+0000","last_active":"2023-11-24T06:25:19.501039+0000","last_peered":"2023-11-24T06:25:19.501039+0000","last_clean":"2023-11-24T06:25:19.501039+0000","last_became_active":"2023-11-24T06:25:17.486300+0000","last_became_peered":"2023-11-24T06:25:17.486300+0000","last_unstale":"2023-11-24T06:25:19.501039+0000","last_undegraded":"2023-11-24T06:25:19.501039+0000","last_fullsized":"2023-11-24T06:25:19.501039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:17.747664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.511179+0000","last_change":"2023-11-24T06:25:18.504808+0000","last_active":"2023-11-24T06:25:18.511179+0000","last_peered":"2023-11-24T06:25:18.511179+0000","last_clean":"2023-11-24T06:25:18.511179+0000","last_became_active":"2023-11-24T06:25:18.504556+0000","last_became_peered":"2023-11-24T06:25:18.504556+0000","last_unstale":"2023-11-24T06:25:18.511179+0000","last_undegraded":"2023-11-24T06:25:18.511179+0000","last_fullsized":"2023-11-24T06:25:18.511179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:42.387098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488323+0000","last_change":"2023-11-24T06:25:17.484744+0000","last_active":"2023-11-24T06:25:18.488323+0000","last_peered":"2023-11-24T06:25:18.488323+0000","last_clean":"2023-11-24T06:25:18.488323+0000","last_became_active":"2023-11-24T06:25:17.484476+0000","last_became_peered":"2023-11-24T06:25:17.484476+0000","last_unstale":"2023-11-24T06:25:18.488323+0000","last_undegraded":"2023-11-24T06:25:18.488323+0000","last_fullsized":"2023-11-24T06:25:18.488323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:25.408123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500738+0000","last_change":"2023-11-24T06:25:18.504231+0000","last_active":"2023-11-24T06:25:19.500738+0000","last_peered":"2023-11-24T06:25:19.500738+0000","last_clean":"2023-11-24T06:25:19.500738+0000","last_became_active":"2023-11-24T06:25:18.503949+0000","last_became_peered":"2023-11-24T06:25:18.503949+0000","last_unstale":"2023-11-24T06:25:19.500738+0000","last_undegraded":"2023-11-24T06:25:19.500738+0000","last_fullsized":"2023-11-24T06:25:19.500738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:46.986918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.492074+0000","last_change":"2023-11-24T06:25:17.489327+0000","last_active":"2023-11-24T06:25:18.492074+0000","last_peered":"2023-11-24T06:25:18.492074+0000","last_clean":"2023-11-24T06:25:18.492074+0000","last_became_active":"2023-11-24T06:25:17.488936+0000","last_became_peered":"2023-11-24T06:25:17.488936+0000","last_unstale":"2023-11-24T06:25:18.492074+0000","last_undegraded":"2023-11-24T06:25:18.492074+0000","last_fullsized":"2023-11-24T06:25:18.492074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:08.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.505660+0000","last_change":"2023-11-24T06:25:18.501849+0000","last_active":"2023-11-24T06:25:18.505660+0000","last_peered":"2023-11-24T06:25:18.505660+0000","last_clean":"2023-11-24T06:25:18.505660+0000","last_became_active":"2023-11-24T06:25:18.499304+0000","last_became_peered":"2023-11-24T06:25:18.499304+0000","last_unstale":"2023-11-24T06:25:18.505660+0000","last_undegraded":"2023-11-24T06:25:18.505660+0000","last_fullsized":"2023-11-24T06:25:18.505660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:12.716038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639273+0000","last_change":"2023-11-24T06:25:17.484655+0000","last_active":"2023-11-24T06:25:19.639273+0000","last_peered":"2023-11-24T06:25:19.639273+0000","last_clean":"2023-11-24T06:25:19.639273+0000","last_became_active":"2023-11-24T06:25:17.484386+0000","last_became_peered":"2023-11-24T06:25:17.484386+0000","last_unstale":"2023-11-24T06:25:19.639273+0000","last_undegraded":"2023-11-24T06:25:19.639273+0000","last_fullsized":"2023-11-24T06:25:19.639273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:13.344722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.507035+0000","last_change":"2023-11-24T06:25:18.501694+0000","last_active":"2023-11-24T06:25:18.507035+0000","last_peered":"2023-11-24T06:25:18.507035+0000","last_clean":"2023-11-24T06:25:18.507035+0000","last_became_active":"2023-11-24T06:25:18.501428+0000","last_became_peered":"2023-11-24T06:25:18.501428+0000","last_unstale":"2023-11-24T06:25:18.507035+0000","last_undegraded":"2023-11-24T06:25:18.507035+0000","last_fullsized":"2023-11-24T06:25:18.507035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:30.284518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688842+0000","last_change":"2023-11-24T06:25:17.486342+0000","last_active":"2023-11-24T06:25:19.688842+0000","last_peered":"2023-11-24T06:25:19.688842+0000","last_clean":"2023-11-24T06:25:19.688842+0000","last_became_active":"2023-11-24T06:25:17.485721+0000","last_became_peered":"2023-11-24T06:25:17.485721+0000","last_unstale":"2023-11-24T06:25:19.688842+0000","last_undegraded":"2023-11-24T06:25:19.688842+0000","last_fullsized":"2023-11-24T06:25:19.688842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:25.824047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688804+0000","last_change":"2023-11-24T06:25:18.510255+0000","last_active":"2023-11-24T06:25:19.688804+0000","last_peered":"2023-11-24T06:25:19.688804+0000","last_clean":"2023-11-24T06:25:19.688804+0000","last_became_active":"2023-11-24T06:25:18.510013+0000","last_became_peered":"2023-11-24T06:25:18.510013+0000","last_unstale":"2023-11-24T06:25:19.688804+0000","last_undegraded":"2023-11-24T06:25:19.688804+0000","last_fullsized":"2023-11-24T06:25:19.688804+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:30.284518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.639122+0000","last_change":"2023-11-24T06:25:17.485977+0000","last_active":"2023-11-24T06:25:19.639122+0000","last_peered":"2023-11-24T06:25:19.639122+0000","last_clean":"2023-11-24T06:25:19.639122+0000","last_became_active":"2023-11-24T06:25:17.485582+0000","last_became_peered":"2023-11-24T06:25:17.485582+0000","last_unstale":"2023-11-24T06:25:19.639122+0000","last_undegraded":"2023-11-24T06:25:19.639122+0000","last_fullsized":"2023-11-24T06:25:19.639122+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:07.711722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500944+0000","last_change":"2023-11-24T06:25:18.502625+0000","last_active":"2023-11-24T06:25:19.500944+0000","last_peered":"2023-11-24T06:25:19.500944+0000","last_clean":"2023-11-24T06:25:19.500944+0000","last_became_active":"2023-11-24T06:25:18.502254+0000","last_became_peered":"2023-11-24T06:25:18.502254+0000","last_unstale":"2023-11-24T06:25:19.500944+0000","last_undegraded":"2023-11-24T06:25:19.500944+0000","last_fullsized":"2023-11-24T06:25:19.500944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:35.536720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688856+0000","last_change":"2023-11-24T06:25:17.487180+0000","last_active":"2023-11-24T06:25:19.688856+0000","last_peered":"2023-11-24T06:25:19.688856+0000","last_clean":"2023-11-24T06:25:19.688856+0000","last_became_active":"2023-11-24T06:25:17.486936+0000","last_became_peered":"2023-11-24T06:25:17.486936+0000","last_unstale":"2023-11-24T06:25:19.688856+0000","last_undegraded":"2023-11-24T06:25:19.688856+0000","last_fullsized":"2023-11-24T06:25:19.688856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:03.170284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.688988+0000","last_change":"2023-11-24T06:25:18.502766+0000","last_active":"2023-11-24T06:25:19.688988+0000","last_peered":"2023-11-24T06:25:19.688988+0000","last_clean":"2023-11-24T06:25:19.688988+0000","last_became_active":"2023-11-24T06:25:18.502457+0000","last_became_peered":"2023-11-24T06:25:18.502457+0000","last_unstale":"2023-11-24T06:25:19.688988+0000","last_undegraded":"2023-11-24T06:25:19.688988+0000","last_fullsized":"2023-11-24T06:25:19.688988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:08.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.638894+0000","last_change":"2023-11-24T06:25:17.484634+0000","last_active":"2023-11-24T06:25:19.638894+0000","last_peered":"2023-11-24T06:25:19.638894+0000","last_clean":"2023-11-24T06:25:19.638894+0000","last_became_active":"2023-11-24T06:25:17.484270+0000","last_became_peered":"2023-11-24T06:25:17.484270+0000","last_unstale":"2023-11-24T06:25:19.638894+0000","last_undegraded":"2023-11-24T06:25:19.638894+0000","last_fullsized":"2023-11-24T06:25:19.638894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:44.193976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498969+0000","last_change":"2023-11-24T06:25:18.519424+0000","last_active":"2023-11-24T06:25:19.498969+0000","last_peered":"2023-11-24T06:25:19.498969+0000","last_clean":"2023-11-24T06:25:19.498969+0000","last_became_active":"2023-11-24T06:25:18.519211+0000","last_became_peered":"2023-11-24T06:25:18.519211+0000","last_unstale":"2023-11-24T06:25:19.498969+0000","last_undegraded":"2023-11-24T06:25:19.498969+0000","last_fullsized":"2023-11-24T06:25:19.498969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:12:46.986918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.488471+0000","last_change":"2023-11-24T06:25:17.487213+0000","last_active":"2023-11-24T06:25:18.488471+0000","last_peered":"2023-11-24T06:25:18.488471+0000","last_clean":"2023-11-24T06:25:18.488471+0000","last_became_active":"2023-11-24T06:25:17.486855+0000","last_became_peered":"2023-11-24T06:25:17.486855+0000","last_unstale":"2023-11-24T06:25:18.488471+0000","last_undegraded":"2023-11-24T06:25:18.488471+0000","last_fullsized":"2023-11-24T06:25:18.488471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:53:23.480844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.517623+0000","last_change":"2023-11-24T06:25:18.516471+0000","last_active":"2023-11-24T06:25:18.517623+0000","last_peered":"2023-11-24T06:25:18.517623+0000","last_clean":"2023-11-24T06:25:18.517623+0000","last_became_active":"2023-11-24T06:25:18.516153+0000","last_became_peered":"2023-11-24T06:25:18.516153+0000","last_unstale":"2023-11-24T06:25:18.517623+0000","last_undegraded":"2023-11-24T06:25:18.517623+0000","last_fullsized":"2023-11-24T06:25:18.517623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:26.834156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.498994+0000","last_change":"2023-11-24T06:25:17.489308+0000","last_active":"2023-11-24T06:25:19.498994+0000","last_peered":"2023-11-24T06:25:19.498994+0000","last_clean":"2023-11-24T06:25:19.498994+0000","last_became_active":"2023-11-24T06:25:17.488994+0000","last_became_peered":"2023-11-24T06:25:17.488994+0000","last_unstale":"2023-11-24T06:25:19.498994+0000","last_undegraded":"2023-11-24T06:25:19.498994+0000","last_fullsized":"2023-11-24T06:25:19.498994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:14.325928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500833+0000","last_change":"2023-11-24T06:25:18.495007+0000","last_active":"2023-11-24T06:25:19.500833+0000","last_peered":"2023-11-24T06:25:19.500833+0000","last_clean":"2023-11-24T06:25:19.500833+0000","last_became_active":"2023-11-24T06:25:18.493873+0000","last_became_peered":"2023-11-24T06:25:18.493873+0000","last_unstale":"2023-11-24T06:25:19.500833+0000","last_undegraded":"2023-11-24T06:25:19.500833+0000","last_fullsized":"2023-11-24T06:25:19.500833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:17.463111+0000","last_clean_scrub_stamp":"2023-11-24T06:25:17.463111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:01.050108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:25:19.500344+0000","last_change":"2023-11-24T06:25:17.488128+0000","last_active":"2023-11-24T06:25:19.500344+0000","last_peered":"2023-11-24T06:25:19.500344+0000","last_clean":"2023-11-24T06:25:19.500344+0000","last_became_active":"2023-11-24T06:25:17.487844+0000","last_became_peered":"2023-11-24T06:25:17.487844+0000","last_unstale":"2023-11-24T06:25:19.500344+0000","last_undegraded":"2023-11-24T06:25:19.500344+0000","last_fullsized":"2023-11-24T06:25:19.500344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:16.453003+0000","last_clean_scrub_stamp":"2023-11-24T06:25:16.453003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:09.802999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:25:18.487813+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:18.487813+0000","last_peered":"2023-11-24T06:25:18.487813+0000","last_clean":"2023-11-24T06:25:18.487813+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:18.487813+0000","last_undegraded":"2023-11-24T06:25:18.487813+0000","last_fullsized":"2023-11-24T06:25:18.487813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.438},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.505}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.622}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.79600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.249},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.27400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.30499999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.622},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.5}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.621}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.434},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.374},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.32700000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:22.979 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:25:22.979 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:25:23.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.137+0000 7f07a1e70700 1 Processor -- start 2023-11-24T06:25:23.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.137+0000 7f07a1e70700 1 -- start start 2023-11-24T06:25:23.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.138+0000 7f07a1e70700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c0746a0 0x7f079c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.138+0000 7f07a1e70700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f079c0afc90 0x7f079c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.138+0000 7f07a1e70700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0b3c60 con 0x7f079c15e160 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.138+0000 7f07a1e70700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c0b4500 con 0x7f079c0afc90 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.138+0000 7f07a1e70700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0b58c0 con 0x7f079c0746a0 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f079c0afc90 0x7f079c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c0746a0 0x7f079c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079b7fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f079c15e160 0x7f079c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:49196/0 (socket says 172.21.15.114:49196) 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079b7fe700 1 -- 172.21.15.114:0/1422299113 learned_addr learned my addr 172.21.15.114:0/1422299113 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f079c0afc90 0x7f079c06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:35554/0 (socket says 172.21.15.114:35554) 2023-11-24T06:25:23.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 -- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c0746a0 msgr2=0x7f079c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:23.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 --2- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c0746a0 0x7f079c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 -- 172.21.15.114:0/1422299113 >> v1:172.21.15.172:6789/0 conn(0x7f079c15e160 legacy=0x7f079c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079a7fc700 1 -- 172.21.15.114:0/1422299113 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251793674 0 0) 0x7f079c0b3c60 con 0x7f079c15e160 2023-11-24T06:25:23.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 -- 172.21.15.114:0/1422299113 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0b0140 con 0x7f079c0afc90 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079bfff700 1 --2- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0afc90 0x7f079c06d930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e633e8781afa58f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079a7fc700 1 -- 172.21.15.114:0/1422299113 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400e070 con 0x7f079c0afc90 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079a7fc700 1 -- 172.21.15.114:0/1422299113 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079400bb20 con 0x7f079c0afc90 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.139+0000 7f079a7fc700 1 -- 172.21.15.114:0/1422299113 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794012430 con 0x7f079c0afc90 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 -- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0afc90 msgr2=0x7f079c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 --2- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0afc90 0x7f079c06d930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 -- 172.21.15.114:0/1422299113 shutdown_connections 2023-11-24T06:25:23.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 --2- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0afc90 0x7f079c06d930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 --2- 172.21.15.114:0/1422299113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c0746a0 0x7f079c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 -- 172.21.15.114:0/1422299113 >> 172.21.15.114:0/1422299113 conn(0x7f079c0d0bf0 msgr2=0x7f079c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:23.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 -- 172.21.15.114:0/1422299113 shutdown_connections 2023-11-24T06:25:23.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.140+0000 7f07a1e70700 1 -- 172.21.15.114:0/1422299113 wait complete. 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.141+0000 7f07a1e70700 1 Processor -- start 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.142+0000 7f07a1e70700 1 -- start start 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 0x7f079c160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 0x7f079c1f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c152300 con 0x7f079c0afc90 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1f3dc0 con 0x7f079c0746a0 2023-11-24T06:25:23.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1f40a0 con 0x7f079c15e160 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39900/0 (socket says 172.21.15.114:39900) 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 -- 172.21.15.114:0/3775380113 learned_addr learned my addr 172.21.15.114:0/3775380113 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079b7fe700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 0x7f079c160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079bfff700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 0x7f079c1f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 -- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 msgr2=0x7f079c1f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 0x7f079c1f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 -- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 msgr2=0x7f079c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 0x7f079c160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 -- 172.21.15.114:0/3775380113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009a70 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079bfff700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 0x7f079c1f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079b7fe700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 0x7f079c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f079affd700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f078c00e9b0 tx=0x7f078c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c009710 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00c850 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00cbc0 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.143+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c1f4380 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.144+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c161310 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.144+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c161530 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.145+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f078c00cd20 con 0x7f079c0afc90 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.145+0000 7f0783fff700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 0x7f078806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.145+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f078c089640 con 0x7f079c0afc90 2023-11-24T06:25:23.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.145+0000 7f079b7fe700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 0x7f078806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.146+0000 7f079b7fe700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 0x7f078806d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0790000f60 tx=0x7f0790007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.150+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f078c054780 con 0x7f079c0afc90 2023-11-24T06:25:23.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.419+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f079c068fb0 con 0x7f079c0afc90 2023-11-24T06:25:23.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.421+0000 7f0783fff700 1 -- 172.21.15.114:0/3775380113 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f078c057da0 con 0x7f079c0afc90 2023-11-24T06:25:23.425 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2023-11-24T06:25:23.425 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g== 2023-11-24T06:25:23.425 INFO:teuthology.orchestra.run.smithi114.stdout: caps mds = "allow" 2023-11-24T06:25:23.425 INFO:teuthology.orchestra.run.smithi114.stdout: caps mon = "allow r" 2023-11-24T06:25:23.425 INFO:teuthology.orchestra.run.smithi114.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:25:23.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 msgr2=0x7f078806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 0x7f078806d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0790000f60 tx=0x7f0790007d30 comp rx=0 tx=0).stop 2023-11-24T06:25:23.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 msgr2=0x7f079c1f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f078c00e9b0 tx=0x7f078c00ed70 comp rx=0 tx=0).stop 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 shutdown_connections 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f078806aed0 0x7f078806d390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f079c15e160 0x7f079c1f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f079c0746a0 0x7f079c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 --2- 172.21.15.114:0/3775380113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f079c0afc90 0x7f079c1f3230 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 >> 172.21.15.114:0/3775380113 conn(0x7f079c0d0bf0 msgr2=0x7f079c15d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 shutdown_connections 2023-11-24T06:25:23.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.424+0000 7f07a1e70700 1 -- 172.21.15.114:0/3775380113 wait complete. 2023-11-24T06:25:23.429 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.0 2023-11-24T06:25:23.439 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.557+0000 7fce42362700 1 Processor -- start 2023-11-24T06:25:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.557+0000 7fce42362700 1 -- start start 2023-11-24T06:25:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce42362700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce3c15c160 0x7fce3c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce42362700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce42362700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c15c540 con 0x7fce3c0b2870 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce42362700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c0b2ee0 con 0x7fce3c15c160 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce42362700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0add40 con 0x7fce3c0b04b0 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce408ff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3bfff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce3c15c160 0x7fce3c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce408ff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:34454/0 (socket says 172.21.15.114:34454) 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce408ff700 1 -- 172.21.15.114:0/3578628172 learned_addr learned my addr 172.21.15.114:0/3578628172 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3b7fe700 1 --1- 172.21.15.114:0/3578628172 >> v1:172.21.15.172:6789/0 conn(0x7fce3c0b2870 0x7fce3c0afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:49212/0 (socket says 172.21.15.114:49212) 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3bfff700 1 -- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 msgr2=0x7fce3c165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3bfff700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3bfff700 1 -- 172.21.15.114:0/3578628172 >> v1:172.21.15.172:6789/0 conn(0x7fce3c0b2870 legacy=0x7fce3c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3bfff700 1 -- 172.21.15.114:0/3578628172 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0ad410 con 0x7fce3c15c160 2023-11-24T06:25:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce3affd700 1 -- 172.21.15.114:0/3578628172 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051908456 0 0) 0x7fce3c15c540 con 0x7fce3c0b2870 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.558+0000 7fce408ff700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce3bfff700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c15c160 0x7fce3c0622a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce30009840 tx=0x7fce30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fb08ee3a2410ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce3affd700 1 -- 172.21.15.114:0/3578628172 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30011830 con 0x7fce3c15c160 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce3affd700 1 -- 172.21.15.114:0/3578628172 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce30011990 con 0x7fce3c15c160 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce3affd700 1 -- 172.21.15.114:0/3578628172 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30011d20 con 0x7fce3c15c160 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 -- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c15c160 msgr2=0x7fce3c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c15c160 0x7fce3c0622a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce30009840 tx=0x7fce30009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 -- 172.21.15.114:0/3578628172 shutdown_connections 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c0b04b0 0x7fce3c165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 --2- 172.21.15.114:0/3578628172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c15c160 0x7fce3c0622a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 -- 172.21.15.114:0/3578628172 >> 172.21.15.114:0/3578628172 conn(0x7fce3c0b57b0 msgr2=0x7fce3c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:23.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 -- 172.21.15.114:0/3578628172 shutdown_connections 2023-11-24T06:25:23.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.559+0000 7fce42362700 1 -- 172.21.15.114:0/3578628172 wait complete. 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.561+0000 7fce42362700 1 Processor -- start 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.561+0000 7fce42362700 1 -- start start 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 0x7fce3c15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c15c160 0x7fce3c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15cb10 con 0x7fce3c0b04b0 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0cae30 con 0x7fce3c0b2870 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce42362700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15ef20 con 0x7fce3c15c160 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39912/0 (socket says 172.21.15.114:39912) 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 -- 172.21.15.114:0/782177032 learned_addr learned my addr 172.21.15.114:0/782177032 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3b7fe700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 0x7fce3c15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 -- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c15c160 msgr2=0x7fce3c14c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce408ff700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c15c160 0x7fce3c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c15c160 0x7fce3c14c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 -- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 msgr2=0x7fce3c15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 0x7fce3c15e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.562+0000 7fce3bfff700 1 -- 172.21.15.114:0/782177032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009b30 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce3b7fe700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 0x7fce3c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce3bfff700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce30009fd0 tx=0x7fce30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce300036a0 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3001f070 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30003a60 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300094a0 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.563+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c15f4a0 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.564+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c15f630 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.564+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce30003bc0 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.564+0000 7fce38ff9700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 0x7fce2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.565+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce30021070 con 0x7fce3c0b04b0 2023-11-24T06:25:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.565+0000 7fce3b7fe700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 0x7fce2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.565+0000 7fce3b7fe700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 0x7fce2806d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007fd0 tx=0x7fce2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:23.572 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.569+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce3c15f630 con 0x7fce3c0b04b0 2023-11-24T06:25:23.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.800+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce3c15f630 con 0x7fce3c0b04b0 2023-11-24T06:25:23.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.801+0000 7fce38ff9700 1 -- 172.21.15.114:0/782177032 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fce3c15f630 con 0x7fce3c0b04b0 2023-11-24T06:25:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:23.805 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:23.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 msgr2=0x7fce2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 0x7fce2806d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007fd0 tx=0x7fce2c009040 comp rx=0 tx=0).stop 2023-11-24T06:25:23.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 msgr2=0x7fce3c15c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:23.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce30009fd0 tx=0x7fce30004750 comp rx=0 tx=0).stop 2023-11-24T06:25:23.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 shutdown_connections 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fce2806aed0 0x7fce2806d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce3c15c160 0x7fce3c14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce3c0b04b0 0x7fce3c15c5d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 --2- 172.21.15.114:0/782177032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce3c0b2870 0x7fce3c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 >> 172.21.15.114:0/782177032 conn(0x7fce3c0b57b0 msgr2=0x7fce3c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 shutdown_connections 2023-11-24T06:25:23.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:23.803+0000 7fce42362700 1 -- 172.21.15.114:0/782177032 wait complete. 2023-11-24T06:25:23.809 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2023-11-24T06:25:24.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:25:17.870023+0000', 'modified': '2023-11-24T06:25:20.896679+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5127': {'gid': 5127, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.114:6837/1960617883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6835', 'nonce': 1960617883}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 1960617883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:25:24.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:25:24.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:25:24.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.340+0000 7fc31a2cb700 1 Processor -- start 2023-11-24T06:25:24.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.340+0000 7fc31a2cb700 1 -- start start 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc31415e080 0x7fc3140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3140af4d0 0x7fc3140afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc31406c000 0x7fc31406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3140af0b0 con 0x7fc31415e080 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31406c9f0 con 0x7fc3140af4d0 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc31a2cb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140b05a0 con 0x7fc31406c000 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc313fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc31415e080 0x7fc3140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc313fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc31415e080 0x7fc3140b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39920/0 (socket says 172.21.15.114:39920) 2023-11-24T06:25:24.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.341+0000 7fc313fff700 1 -- 172.21.15.114:0/825153128 learned_addr learned my addr 172.21.15.114:0/825153128 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 -- 172.21.15.114:0/825153128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc31406c000 msgr2=0x7fc31406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 --2- 172.21.15.114:0/825153128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc31406c000 0x7fc31406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 -- 172.21.15.114:0/825153128 >> v2:172.21.15.172:3300/0 conn(0x7fc3140af4d0 msgr2=0x7fc3140afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 --2- 172.21.15.114:0/825153128 >> v2:172.21.15.172:3300/0 conn(0x7fc3140af4d0 0x7fc3140afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 -- 172.21.15.114:0/825153128 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140ae430 con 0x7fc31415e080 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc313fff700 1 --2- 172.21.15.114:0/825153128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc3140b5050 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc31406e4e0 tx=0x7fc304009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd5e5feca886521 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc312ffd700 1 -- 172.21.15.114:0/825153128 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400b870 con 0x7fc31415e080 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc312ffd700 1 -- 172.21.15.114:0/825153128 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30400b9d0 con 0x7fc31415e080 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc312ffd700 1 -- 172.21.15.114:0/825153128 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30400bd60 con 0x7fc31415e080 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc31a2cb700 1 -- 172.21.15.114:0/825153128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 msgr2=0x7fc3140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.342+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/825153128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc3140b5050 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc31406e4e0 tx=0x7fc304009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 -- 172.21.15.114:0/825153128 shutdown_connections 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/825153128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc31406c000 0x7fc31406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/825153128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc3140b5050 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/825153128 >> v2:172.21.15.172:3300/0 conn(0x7fc3140af4d0 0x7fc3140afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 -- 172.21.15.114:0/825153128 >> 172.21.15.114:0/825153128 conn(0x7fc314071b90 msgr2=0x7fc314071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 -- 172.21.15.114:0/825153128 shutdown_connections 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.343+0000 7fc31a2cb700 1 -- 172.21.15.114:0/825153128 wait complete. 2023-11-24T06:25:24.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.344+0000 7fc31a2cb700 1 Processor -- start 2023-11-24T06:25:24.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.344+0000 7fc31a2cb700 1 -- start start 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31406c000 0x7fc3141568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3140af4d0 0x7fc3141490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3141561d0 con 0x7fc31415e080 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31415f380 con 0x7fc31406c000 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31415e800 con 0x7fc3140af4d0 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39930/0 (socket says 172.21.15.114:39930) 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 -- 172.21.15.114:0/1117971349 learned_addr learned my addr 172.21.15.114:0/1117971349 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.352 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc313fff700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31406c000 0x7fc3141568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc3137fe700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3140af4d0 0x7fc3141490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 -- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3140af4d0 msgr2=0x7fc3141490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3140af4d0 0x7fc3141490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 -- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31406c000 msgr2=0x7fc3141568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31406c000 0x7fc3141568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 -- 172.21.15.114:0/1117971349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304009550 con 0x7fc31415e080 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc318868700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00b970 tx=0x7fc30c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc310ff9700 1 -- 172.21.15.114:0/1117971349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00c9b0 con 0x7fc31415e080 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc310ff9700 1 -- 172.21.15.114:0/1117971349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c00cb30 con 0x7fc31415e080 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31415eae0 con 0x7fc31415e080 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc310ff9700 1 -- 172.21.15.114:0/1117971349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c014440 con 0x7fc31415e080 2023-11-24T06:25:24.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.345+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31415f0a0 con 0x7fc31415e080 2023-11-24T06:25:24.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.347+0000 7fc310ff9700 1 -- 172.21.15.114:0/1117971349 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc30c01d3b0 con 0x7fc31415e080 2023-11-24T06:25:24.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.347+0000 7fc310ff9700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 0x7fc30006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.347+0000 7fc310ff9700 1 -- 172.21.15.114:0/1117971349 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc30c006e40 con 0x7fc31415e080 2023-11-24T06:25:24.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.347+0000 7fc313fff700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 0x7fc30006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.348+0000 7fc313fff700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 0x7fc30006d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc304014040 tx=0x7fc304008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.349+0000 7fc3264c4b80 1 Processor -- start 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.349+0000 7fc3264c4b80 1 -- start start 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bee840 0x563151bec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151ba2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151b1b770 con 0x563151bbded0 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151b02750 con 0x563151beec20 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563151b00070 con 0x563151bee840 2023-11-24T06:25:24.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39936/0 (socket says 172.21.15.114:39936) 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 -- 172.21.15.114:0/3614020697 learned_addr learned my addr 172.21.15.114:0/3614020697 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 -- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bee840 msgr2=0x563151bec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bee840 0x563151bec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 -- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 msgr2=0x563151ba2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151ba2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 -- 172.21.15.114:0/3614020697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631519785e0 con 0x563151bbded0 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc308007820 tx=0x7fc308007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce6546db7c91118 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.350+0000 7fc2f9ffb700 1 -- 172.21.15.114:0/3614020697 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080125f0 con 0x563151bbded0 2023-11-24T06:25:24.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc2f9ffb700 1 -- 172.21.15.114:0/3614020697 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308012750 con 0x563151bbded0 2023-11-24T06:25:24.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc2f9ffb700 1 -- 172.21.15.114:0/3614020697 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308012ac0 con 0x563151bbded0 2023-11-24T06:25:24.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3614020697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 msgr2=0x563151bead20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc308007820 tx=0x7fc308007b30 comp rx=0 tx=0).stop 2023-11-24T06:25:24.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3614020697 shutdown_connections 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151ba2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bee840 0x563151bec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3614020697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151bead20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3614020697 >> 172.21.15.114:0/3614020697 conn(0x563151ad6310 msgr2=0x563151bece70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3614020697 shutdown_connections 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.351+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3614020697 wait complete. 2023-11-24T06:25:24.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.352+0000 7fc3264c4b80 1 -- start start 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.352+0000 7fc3264c4b80 1 Processor -- start 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151bd3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151be1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151afb960 con 0x563151bee840 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151ab5510 con 0x563151beec20 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563151ab51b0 con 0x563151bbded0 2023-11-24T06:25:24.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 10 client.0 _ll_register_callbacks cb 0x5631519e0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39942/0 (socket says 172.21.15.114:39942) 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 -- 172.21.15.114:0/3431097004 learned_addr learned my addr 172.21.15.114:0/3431097004 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc318868700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151be1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc313fff700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151bd3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 -- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 msgr2=0x563151bd3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151bd3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 -- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 msgr2=0x563151be1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151be1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 -- 172.21.15.114:0/3431097004 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30800e040 con 0x563151bee840 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3137fe700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc308015040 tx=0x7fc308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec2fdf90c5632f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc2f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30800a040 con 0x563151bee840 2023-11-24T06:25:24.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc308007000 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.353+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563151b9aa40 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc308012e50 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 10 client.5310 fetch_fsmap learned FSMap version 17 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563151924d90 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563151b9aa40 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc308013690 con 0x563151bee840 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 10 client.5310 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 10 client.5310 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:25:24.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 20 client.5310 populate_metadata read hostname 'smithi114' 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151c08c50 0x563151bdf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 4 client.5310 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 --> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563151b76840 con 0x563151c08c50 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc313fff700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151c08c50 0x563151bdf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc2f2ffd700 10 client.5310 ms_handle_connect on v2:172.21.15.114:6835/1960617883 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.354+0000 7fc313fff700 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151c08c50 0x563151bdf950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba24a0340c93b728 server_cookie=721df929532bffcc in_seq=0 out_seq=0 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/3431097004 <== mds.0 v2:172.21.15.114:6835/1960617883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563151b76840 con 0x563151c08c50 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc2f2ffd700 10 client.5310 handle_command_reply: tid=0 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 client.5310 shutdown 2023-11-24T06:25:24.371 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 msgr2=0x563151be1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc308015040 tx=0x7fc308004750 comp rx=0 tx=0).stop 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 shutdown_connections 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151c08c50 0x563151bdf950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151beec20 0x563151be1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc2f2ffd700 0 client.5310 ms_handle_reset on v2:172.21.15.114:6835/1960617883 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151bd3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/3431097004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bee840 0x563151be1340 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 >> 172.21.15.114:0/3431097004 conn(0x563151ad6310 msgr2=0x563151b9c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 shutdown_connections 2023-11-24T06:25:24.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 1 -- 172.21.15.114:0/3431097004 wait complete. 2023-11-24T06:25:24.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.355+0000 7fc3264c4b80 20 client.5310 trim_cache size 0 max 16384 2023-11-24T06:25:24.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.373+0000 7fc3264c4b80 1 Processor -- start 2023-11-24T06:25:24.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.373+0000 7fc3264c4b80 1 -- start start 2023-11-24T06:25:24.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151be23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151b9c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56315197a8d0 con 0x563151beec20 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151ba99d0 con 0x563151bee840 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563151ba3ea0 con 0x563151bbded0 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39952/0 (socket says 172.21.15.114:39952) 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 -- 172.21.15.114:0/1548429514 learned_addr learned my addr 172.21.15.114:0/1548429514 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc3137fe700 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151b9c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc313fff700 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151be23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 -- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 msgr2=0x563151be23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151be23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 -- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 msgr2=0x563151b9c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151b9c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.374+0000 7fc318868700 1 -- 172.21.15.114:0/1548429514 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56315190dc10 con 0x563151beec20 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc318868700 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00be50 tx=0x7fc30c057a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230c10ca4e14dfa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc2fa7fc700 1 -- 172.21.15.114:0/1548429514 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c05c280 con 0x563151beec20 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc2fa7fc700 1 -- 172.21.15.114:0/1548429514 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c060430 con 0x563151beec20 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc2fa7fc700 1 -- 172.21.15.114:0/1548429514 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c058390 con 0x563151beec20 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 -- 172.21.15.114:0/1548429514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 msgr2=0x563151b9ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00be50 tx=0x7fc30c057a90 comp rx=0 tx=0).stop 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 -- 172.21.15.114:0/1548429514 shutdown_connections 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151beec20 0x563151b9ce30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151bbded0 0x563151be23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/1548429514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151b9c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 -- 172.21.15.114:0/1548429514 >> 172.21.15.114:0/1548429514 conn(0x563151ad6310 msgr2=0x563151bec0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 -- 172.21.15.114:0/1548429514 shutdown_connections 2023-11-24T06:25:24.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.375+0000 7fc3264c4b80 1 -- 172.21.15.114:0/1548429514 wait complete. 2023-11-24T06:25:24.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.376+0000 7fc3264c4b80 1 -- start start 2023-11-24T06:25:24.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.376+0000 7fc3264c4b80 1 Processor -- start 2023-11-24T06:25:24.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151c040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151beec20 0x563151c07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151b9cc70 con 0x563151bbded0 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563151beaea0 con 0x563151bee840 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563151c03a50 con 0x563151beec20 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3264c4b80 10 client.0 _ll_register_callbacks cb 0x5631519e0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39954/0 (socket says 172.21.15.114:39954) 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 -- 172.21.15.114:0/46859841 learned_addr learned my addr 172.21.15.114:0/46859841 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc3137fe700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151c040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc318868700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151beec20 0x563151c07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 -- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151beec20 msgr2=0x563151c07420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151beec20 0x563151c07420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 -- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 msgr2=0x563151c040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151c040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 -- 172.21.15.114:0/46859841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31415f0a0 con 0x563151bbded0 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc313fff700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc304009fd0 tx=0x7fc30400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ed6a816c4bd2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.377+0000 7fc2f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30402d2f0 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc30402a040 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc308012750 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc30402d450 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 10 client.5316 fetch_fsmap learned FSMap version 17 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x56315194b200 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308012750 con 0x563151bbded0 2023-11-24T06:25:24.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc304033070 con 0x563151bbded0 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 10 client.5316 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 10 client.5316 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 20 client.5316 populate_metadata read hostname 'smithi114' 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151b9c060 0x563151bf03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 4 client.5316 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 --> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563151ac1f80 con 0x563151b9c060 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3137fe700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151b9c060 0x563151bf03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc2f2ffd700 10 client.5316 ms_handle_connect on v2:172.21.15.114:6835/1960617883 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.378+0000 7fc3137fe700 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151b9c060 0x563151bf03f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272ed7abcbe7c590 server_cookie=5380021f2417d635 in_seq=0 out_seq=0 2023-11-24T06:25:24.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc2f2ffd700 1 -- 172.21.15.114:0/46859841 <== mds.0 v2:172.21.15.114:6835/1960617883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563151ac1f80 con 0x563151b9c060 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc2f2ffd700 10 client.5316 handle_command_reply: tid=0 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 client.5316 shutdown 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 msgr2=0x563151c5e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc304009fd0 tx=0x7fc30400e000 comp rx=0 tx=0).stop 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 shutdown_connections 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:6835/1960617883,v1:172.21.15.114:6837/1960617883] conn(0x563151b9c060 0x563151bf03f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563151beec20 0x563151c07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc2f2ffd700 0 client.5316 ms_handle_reset on v2:172.21.15.114:6835/1960617883 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563151bbded0 0x563151c5e810 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 --2- 172.21.15.114:0/46859841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563151bee840 0x563151c040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 >> 172.21.15.114:0/46859841 conn(0x563151ad6310 msgr2=0x563151be23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 shutdown_connections 2023-11-24T06:25:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 1 -- 172.21.15.114:0/46859841 wait complete. 2023-11-24T06:25:24.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc3264c4b80 20 client.5316 trim_cache size 0 max 16384 2023-11-24T06:25:24.393 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:25:24.394 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "cfb0acb2-02f2-43f9-bba4-c35657dcf6fd", 2023-11-24T06:25:24.394 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2023-11-24T06:25:24.394 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 5127, 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 5.8887402050000004, 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 17, 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 34, 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 34, 2023-11-24T06:25:24.395 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 13.898748973 2023-11-24T06:25:24.396 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:25:24.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 msgr2=0x7fc30006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 0x7fc30006d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc304014040 tx=0x7fc304008030 comp rx=0 tx=0).stop 2023-11-24T06:25:24.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 msgr2=0x7fc314158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00b970 tx=0x7fc30c00bd30 comp rx=0 tx=0).stop 2023-11-24T06:25:24.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 shutdown_connections 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc30006aed0 0x7fc30006d390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.379+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc31415e080 0x7fc314158a60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.380+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31406c000 0x7fc3141568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.380+0000 7fc31a2cb700 1 --2- 172.21.15.114:0/1117971349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3140af4d0 0x7fc3141490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.380+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 >> 172.21.15.114:0/1117971349 conn(0x7fc314071b90 msgr2=0x7fc3140b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.380+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 shutdown_connections 2023-11-24T06:25:24.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.380+0000 7fc31a2cb700 1 -- 172.21.15.114:0/1117971349 wait complete. 2023-11-24T06:25:24.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:25:24.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.518+0000 7f5bd52ba700 1 Processor -- start 2023-11-24T06:25:24.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.518+0000 7f5bd52ba700 1 -- start start 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bd52ba700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bd015c160 0x7f5bd00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bd52ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bd52ba700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd015c540 con 0x7f5bd00b2870 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bd52ba700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd00b2ee0 con 0x7f5bd015c160 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bd52ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00add40 con 0x7f5bd00b04b0 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bd015c160 0x7f5bd00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bce7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5bd00b2870 0x7f5bd00afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55672/0 (socket says 172.21.15.114:55672) 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bd015c160 0x7f5bd00622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39962/0 (socket says 172.21.15.114:39962) 2023-11-24T06:25:24.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bce7fc700 1 -- 172.21.15.114:0/2892179125 learned_addr learned my addr 172.21.15.114:0/2892179125 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 -- 172.21.15.114:0/2892179125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 msgr2=0x7f5bd0165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bcf7fe700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bcdffb700 1 -- 172.21.15.114:0/2892179125 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296318380 0 0) 0x7f5bd015c540 con 0x7f5bd00b2870 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 -- 172.21.15.114:0/2892179125 >> v1:172.21.15.114:6789/0 conn(0x7f5bd00b2870 legacy=0x7f5bd00afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bceffd700 1 -- 172.21.15.114:0/2892179125 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00ad410 con 0x7f5bd015c160 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.519+0000 7f5bcf7fe700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bceffd700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd015c160 0x7f5bd00622a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009840 tx=0x7f5bc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91cc40cbaa7246c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bcdffb700 1 -- 172.21.15.114:0/2892179125 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4011830 con 0x7f5bd015c160 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bcdffb700 1 -- 172.21.15.114:0/2892179125 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4011990 con 0x7f5bd015c160 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bcdffb700 1 -- 172.21.15.114:0/2892179125 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4011d20 con 0x7f5bd015c160 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2892179125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd015c160 msgr2=0x7f5bd00622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd015c160 0x7f5bd00622a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009840 tx=0x7f5bc4009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2892179125 shutdown_connections 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2892179125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd015c160 0x7f5bd00622a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2892179125 >> 172.21.15.114:0/2892179125 conn(0x7f5bd00b57b0 msgr2=0x7f5bd00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2892179125 shutdown_connections 2023-11-24T06:25:24.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.520+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2892179125 wait complete. 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.522+0000 7f5bd52ba700 1 Processor -- start 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.522+0000 7f5bd52ba700 1 -- start start 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd015c160 0x7f5bd015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd015a2b0 con 0x7f5bd00b2870 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0158a80 con 0x7f5bd015c160 2023-11-24T06:25:24.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bd52ba700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0158cd0 con 0x7f5bd00b04b0 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39970/0 (socket says 172.21.15.114:39970) 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 -- 172.21.15.114:0/2844467318 learned_addr learned my addr 172.21.15.114:0/2844467318 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bceffd700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bcf7fe700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd015c160 0x7f5bd015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 -- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 msgr2=0x7f5bd0159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 -- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd015c160 msgr2=0x7f5bd015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd015c160 0x7f5bd015a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 -- 172.21.15.114:0/2844467318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40094a0 con 0x7f5bd00b2870 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.523+0000 7f5bce7fc700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c550 tx=0x7f5bc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004c80 con 0x7f5bd00b2870 2023-11-24T06:25:24.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc0004e00 con 0x7f5bd00b2870 2023-11-24T06:25:24.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0158fb0 con 0x7f5bd00b2870 2023-11-24T06:25:24.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0007310 con 0x7f5bd00b2870 2023-11-24T06:25:24.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd0159570 con 0x7f5bd00b2870 2023-11-24T06:25:24.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.524+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0067f60 con 0x7f5bd00b2870 2023-11-24T06:25:24.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.525+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bc0007470 con 0x7f5bd00b2870 2023-11-24T06:25:24.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.526+0000 7f5bb77fe700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 0x7f5bbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:24.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.526+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc0089910 con 0x7f5bd00b2870 2023-11-24T06:25:24.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.526+0000 7f5bceffd700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 0x7f5bbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:24.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.526+0000 7f5bceffd700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 0x7f5bbc06d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000c00 tx=0x7f5bc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:24.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.530+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bc0054aa0 con 0x7f5bd00b2870 2023-11-24T06:25:24.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.766+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5bd00b0c50 con 0x7f5bd00b2870 2023-11-24T06:25:24.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.767+0000 7f5bb77fe700 1 -- 172.21.15.114:0/2844467318 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5bc00580c0 con 0x7f5bd00b2870 2023-11-24T06:25:24.770 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:25:24.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 msgr2=0x7f5bbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 0x7f5bbc06d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000c00 tx=0x7f5bc4008030 comp rx=0 tx=0).stop 2023-11-24T06:25:24.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 msgr2=0x7f5bd015eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c550 tx=0x7f5bc000c910 comp rx=0 tx=0).stop 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 shutdown_connections 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bbc06aed0 0x7f5bbc06d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bd015c160 0x7f5bd015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bd00b04b0 0x7f5bd0159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 --2- 172.21.15.114:0/2844467318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bd00b2870 0x7f5bd015eeb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 >> 172.21.15.114:0/2844467318 conn(0x7f5bd00b57b0 msgr2=0x7f5bd00af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 shutdown_connections 2023-11-24T06:25:24.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:24.769+0000 7f5bd52ba700 1 -- 172.21.15.114:0/2844467318 wait complete. 2023-11-24T06:25:24.784 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:25:24.784 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:25:24.784 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2023-11-24T06:25:24.784 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:25:24.784 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:25:24.785 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:25:24.785 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:25:24.785 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:25:24.785 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:25:24.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:24.785 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2023-11-24T06:25:24.802 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:24.803 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::9ca9:e760:b5c4:9eac/64 scope link noprefixroute 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::385c:94ff:fec5:c8f4/64 scope link 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.804 INFO:teuthology.orchestra.run.smithi114.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether fa:2a:57:62:51:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::f82a:57ff:fe62:518a/64 scope link 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::acf4:a2ff:fe10:15ae/64 scope link 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:24.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:24.805 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:25:24.868 INFO:teuthology.orchestra.run.smithi114.stdout:1 2023-11-24T06:25:24.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:24.870 DEBUG:teuthology.orchestra.run.smithi114:> route 2023-11-24T06:25:24.928 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2023-11-24T06:25:24.929 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:25:24.929 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:25:24.929 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:25:24.929 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:25:24.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:24.929 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:25:24.930 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:25:24.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:25:24.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:25:24.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:25:24.930 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:25:25.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:25.034 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:25:25.051 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:25:25.051 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:25:25.051 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:25:25.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:25.051 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2023-11-24T06:25:25.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:25:25.118 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:25:25.175 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:25:25.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:25:25.175 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:25:25.231 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:25:25.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:25.232 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:25.317 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:25.318 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:25:25.345 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:25:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:25:25.422 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:25:25.424 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:25:25.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:25.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:25:25.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:25.451 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:25:25.523 INFO:teuthology.orchestra.run.smithi114.stdout:client4799 2023-11-24T06:25:25.524 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:25.524 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:25.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:25:25.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.739+0000 7f1ef1d18700 1 Processor -- start 2023-11-24T06:25:25.742 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.739+0000 7f1ef1d18700 1 -- start start 2023-11-24T06:25:25.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1ef1d18700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1eec0af4d0 0x7f1eec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:25.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1ef1d18700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0ae430 con 0x7f1eec15e080 2023-11-24T06:25:25.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1ef1d18700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0ae710 con 0x7f1eec06c710 2023-11-24T06:25:25.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1ef1d18700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec0af0b0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eebfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1eec0af4d0 0x7f1eec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeb7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1eec15e080 0x7f1eec06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55688/0 (socket says 172.21.15.114:55688) 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeb7fe700 1 -- 172.21.15.114:0/2364699450 learned_addr learned my addr 172.21.15.114:0/2364699450 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eebfff700 1 -- 172.21.15.114:0/2364699450 >> v1:172.21.15.172:6789/0 conn(0x7f1eec06c710 legacy=0x7f1eec06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eebfff700 1 -- 172.21.15.114:0/2364699450 >> v1:172.21.15.114:6789/0 conn(0x7f1eec15e080 legacy=0x7f1eec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eebfff700 1 -- 172.21.15.114:0/2364699450 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0ae9f0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeaffd700 1 -- 172.21.15.114:0/2364699450 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888801421 0 0) 0x7f1eec0ae430 con 0x7f1eec15e080 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eebfff700 1 --2- 172.21.15.114:0/2364699450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec0b5050 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009d90 tx=0x7f1ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455c2748b4bec233 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeaffd700 1 -- 172.21.15.114:0/2364699450 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400e070 con 0x7f1eec0af4d0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeaffd700 1 -- 172.21.15.114:0/2364699450 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400bbd0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.740+0000 7f1eeaffd700 1 -- 172.21.15.114:0/2364699450 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4012430 con 0x7f1eec0af4d0 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2364699450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 msgr2=0x7f1eec0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2364699450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec0b5050 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009d90 tx=0x7f1ee400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:25.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2364699450 shutdown_connections 2023-11-24T06:25:25.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2364699450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec0b5050 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2364699450 >> 172.21.15.114:0/2364699450 conn(0x7f1eec071b90 msgr2=0x7f1eec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:25.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2364699450 shutdown_connections 2023-11-24T06:25:25.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.741+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2364699450 wait complete. 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.742+0000 7f1ef1d18700 1 Processor -- start 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.742+0000 7f1ef1d18700 1 -- start start 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 0x7f1eec153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 0x7f1eec141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:25.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1542a0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec15f4b0 con 0x7f1eec15e080 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ef1d18700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec15f750 con 0x7f1eec06c710 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39988/0 (socket says 172.21.15.114:39988) 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 -- 172.21.15.114:0/2744561275 learned_addr learned my addr 172.21.15.114:0/2744561275 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1eebfff700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 0x7f1eec141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1eeb7fe700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 0x7f1eec153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 -- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 msgr2=0x7f1eec153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 0x7f1eec153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 -- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 msgr2=0x7f1eec141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 0x7f1eec141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1ee3fff700 1 -- 172.21.15.114:0/2744561275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee4009a70 con 0x7f1eec0af4d0 2023-11-24T06:25:25.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1eeb7fe700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 0x7f1eec153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.743+0000 7f1eebfff700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 0x7f1eec141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ee3fff700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c4e0 tx=0x7f1ed400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40052d0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4005430 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40038c0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec15fa30 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.744+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec15fff0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.745+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec0753f0 con 0x7f1eec0af4d0 2023-11-24T06:25:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.745+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ed4003a20 con 0x7f1eec0af4d0 2023-11-24T06:25:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.745+0000 7f1ee8ff9700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 0x7f1ecc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.745+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ed4088b40 con 0x7f1eec0af4d0 2023-11-24T06:25:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.745+0000 7f1eeb7fe700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 0x7f1ecc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.746+0000 7f1eeb7fe700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 0x7f1ecc06d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000f60 tx=0x7f1edc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.750+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ed4053b60 con 0x7f1eec0af4d0 2023-11-24T06:25:25.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.989+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1eec067f60 con 0x7f1eec0af4d0 2023-11-24T06:25:25.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.989+0000 7f1ee8ff9700 1 -- 172.21.15.114:0/2744561275 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ed4017020 con 0x7f1eec0af4d0 2023-11-24T06:25:25.993 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 msgr2=0x7f1ecc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 0x7f1ecc06d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000f60 tx=0x7f1edc009040 comp rx=0 tx=0).stop 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 msgr2=0x7f1eec1607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c4e0 tx=0x7f1ed400c8a0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 shutdown_connections 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ecc06aed0 0x7f1ecc06d390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15e080 0x7f1eec141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eec06c710 0x7f1eec153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 --2- 172.21.15.114:0/2744561275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eec0af4d0 0x7f1eec1607f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:25.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 >> 172.21.15.114:0/2744561275 conn(0x7f1eec071b90 msgr2=0x7f1eec0ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:25.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 shutdown_connections 2023-11-24T06:25:25.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:25.991+0000 7f1ef1d18700 1 -- 172.21.15.114:0/2744561275 wait complete. 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:25:26.007 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:25:26.008 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:25:26.008 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:25:26.008 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:25:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.008 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-11-24T06:25:26.030 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:25:26.030 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:26.031 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42354sec preferred_lft 42354sec 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c36d:1855:677d:d218/64 scope link noprefixroute 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:25:26.032 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::4cb3:adff:fe77:17e6/64 scope link 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:25:26.033 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::3838:94ff:fea3:1687/64 scope link 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 56:7e:24:58:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::547e:24ff:fe58:9cd/64 scope link 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:25:26.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.034 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:25:26.099 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:25:26.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.101 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-11-24T06:25:26.162 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-11-24T06:25:26.163 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:25:26.163 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:25:26.163 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:25:26.164 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:25:26.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.164 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:25:26.165 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:25:26.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:25:26.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:25:26.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:25:26.165 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:25:26.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.273 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:25:26.289 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:25:26.290 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:25:26.290 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:25:26.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.290 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-11-24T06:25:26.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:25:26.359 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:25:26.416 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:25:26.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:25:26.416 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:25:26.473 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:25:26.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.473 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:26.560 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:26.560 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:25:26.584 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:25:26.585 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:25:26.585 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:25:26.660 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:25:26.660 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:25:26.660 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:25:26.661 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:25:26.662 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:25:26.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.663 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:25:26.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:26.689 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:25:26.762 INFO:teuthology.orchestra.run.smithi172.stdout:client5337 2023-11-24T06:25:26.762 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:26.762 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:26.821 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:25:26.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.955+0000 7f698fd8e700 1 Processor -- start 2023-11-24T06:25:26.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.955+0000 7f698fd8e700 1 -- start start 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698fd8e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 0x7f698806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698fd8e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f69880afb60 0x7f698806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698fd8e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69881624c0 con 0x7f69880b5b60 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698fd8e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698815e490 con 0x7f69880afb60 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698fd8e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698806ebc0 con 0x7f698815e0b0 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f69880afb60 0x7f698806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f69880afb60 0x7f698806c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39996/0 (socket says 172.21.15.114:39996) 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 -- 172.21.15.114:0/1067865491 learned_addr learned my addr 172.21.15.114:0/1067865491 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698db2a700 1 --2- 172.21.15.114:0/1067865491 >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 0x7f698806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 -- 172.21.15.114:0/1067865491 >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 msgr2=0x7f698806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 --2- 172.21.15.114:0/1067865491 >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 0x7f698806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 -- 172.21.15.114:0/1067865491 >> v1:172.21.15.172:6789/0 conn(0x7f69880b5b60 legacy=0x7f6988073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 -- 172.21.15.114:0/1067865491 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988161b90 con 0x7f69880afb60 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698db2a700 1 --2- 172.21.15.114:0/1067865491 >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 0x7f698806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.956+0000 7f698e32b700 1 --2- 172.21.15.114:0/1067865491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880afb60 0x7f698806c540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6988074ff0 tx=0x7f698000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca6d67d27fea502 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698cb28700 1 -- 172.21.15.114:0/1067865491 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000e070 con 0x7f69880afb60 2023-11-24T06:25:26.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698cb28700 1 -- 172.21.15.114:0/1067865491 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6980003a20 con 0x7f69880afb60 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698cb28700 1 -- 172.21.15.114:0/1067865491 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980003dd0 con 0x7f69880afb60 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 -- 172.21.15.114:0/1067865491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880afb60 msgr2=0x7f698806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1067865491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880afb60 0x7f698806c540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6988074ff0 tx=0x7f698000b330 comp rx=0 tx=0).stop 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 -- 172.21.15.114:0/1067865491 shutdown_connections 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1067865491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880afb60 0x7f698806c540 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1067865491 >> v2:172.21.15.172:3300/0 conn(0x7f698815e0b0 0x7f698806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 -- 172.21.15.114:0/1067865491 >> 172.21.15.114:0/1067865491 conn(0x7f6988071b50 msgr2=0x7f6988071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 -- 172.21.15.114:0/1067865491 shutdown_connections 2023-11-24T06:25:26.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.957+0000 7f698fd8e700 1 -- 172.21.15.114:0/1067865491 wait complete. 2023-11-24T06:25:26.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.958+0000 7f698fd8e700 1 Processor -- start 2023-11-24T06:25:26.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 -- start start 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69880afb60 0x7f6988144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f698815e0b0 0x7f6988158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881f4480 con 0x7f69880b5b60 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988160bd0 con 0x7f69880afb60 2023-11-24T06:25:26.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698fd8e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69881f3b50 con 0x7f698815e0b0 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698d329700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698d329700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40000/0 (socket says 172.21.15.114:40000) 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.959+0000 7f698d329700 1 -- 172.21.15.114:0/1519172980 learned_addr learned my addr 172.21.15.114:0/1519172980 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698db2a700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69880afb60 0x7f6988144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698e32b700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f698815e0b0 0x7f6988158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 -- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f698815e0b0 msgr2=0x7f6988158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f698815e0b0 0x7f6988158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 -- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69880afb60 msgr2=0x7f6988144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69880afb60 0x7f6988144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 -- 172.21.15.114:0/1519172980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009b30 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698d329700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f697800c520 tx=0x7f697800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978016070 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978006860 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69881f3e30 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978006bf0 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.960+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69881f4910 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.961+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69880624c0 con 0x7f69880b5b60 2023-11-24T06:25:26.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.961+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6978006d50 con 0x7f69880b5b60 2023-11-24T06:25:26.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.962+0000 7f697e7fc700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 0x7f696c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:26.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.962+0000 7f698db2a700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 0x7f696c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:26.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.962+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6978088da0 con 0x7f69880b5b60 2023-11-24T06:25:26.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.962+0000 7f698db2a700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 0x7f696c06d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:26.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:26.967+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6978053e50 con 0x7f69880b5b60 2023-11-24T06:25:27.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.200+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6988067f60 con 0x7f69880b5b60 2023-11-24T06:25:27.852 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.848+0000 7f697e7fc700 1 -- 172.21.15.114:0/1519172980 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6978057470 con 0x7f69880b5b60 2023-11-24T06:25:27.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 msgr2=0x7f696c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 0x7f696c06d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 msgr2=0x7f69881601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f697800c520 tx=0x7f697800c8e0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 shutdown_connections 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f696c06aed0 0x7f696c06d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f698815e0b0 0x7f6988158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69880afb60 0x7f6988144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 --2- 172.21.15.114:0/1519172980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69880b5b60 0x7f69881601a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 >> 172.21.15.114:0/1519172980 conn(0x7f6988071b50 msgr2=0x7f6988073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:27.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 shutdown_connections 2023-11-24T06:25:27.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.851+0000 7f698fd8e700 1 -- 172.21.15.114:0/1519172980 wait complete. 2023-11-24T06:25:27.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:25:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.987+0000 7f4530d3a700 1 Processor -- start 2023-11-24T06:25:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.987+0000 7f4530d3a700 1 -- start start 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f4530d3a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f452c0d1b00 0x7f452c0d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f4530d3a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f452c0b1cb0 0x7f452c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f4530d3a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c0ce4a0 con 0x7f452c15e110 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f4530d3a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0d2490 con 0x7f452c0b1cb0 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f4530d3a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0cefe0 con 0x7f452c0d1b00 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f452ad9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f452c0b1cb0 0x7f452c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f452a59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f452c15e110 0x7f452c0b1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55714/0 (socket says 172.21.15.114:55714) 2023-11-24T06:25:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.988+0000 7f452a59c700 1 -- 172.21.15.114:0/825041324 learned_addr learned my addr 172.21.15.114:0/825041324 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f4529d9b700 1 --2- 172.21.15.114:0/825041324 >> v2:172.21.15.172:3300/0 conn(0x7f452c0d1b00 0x7f452c0d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452ad9d700 1 -- 172.21.15.114:0/825041324 >> v2:172.21.15.172:3300/0 conn(0x7f452c0d1b00 msgr2=0x7f452c0d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452ad9d700 1 --2- 172.21.15.114:0/825041324 >> v2:172.21.15.172:3300/0 conn(0x7f452c0d1b00 0x7f452c0d1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452ad9d700 1 -- 172.21.15.114:0/825041324 >> v1:172.21.15.114:6789/0 conn(0x7f452c15e110 legacy=0x7f452c0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452ad9d700 1 -- 172.21.15.114:0/825041324 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0cf470 con 0x7f452c0b1cb0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452959a700 1 -- 172.21.15.114:0/825041324 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301448235 0 0) 0x7f452c0ce4a0 con 0x7f452c15e110 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452ad9d700 1 --2- 172.21.15.114:0/825041324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c0b2090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f452c09b7c0 tx=0x7f452400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e27cee4f683aa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452959a700 1 -- 172.21.15.114:0/825041324 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400e070 con 0x7f452c0b1cb0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452959a700 1 -- 172.21.15.114:0/825041324 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452400bbd0 con 0x7f452c0b1cb0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f452959a700 1 -- 172.21.15.114:0/825041324 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524012430 con 0x7f452c0b1cb0 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.989+0000 7f4530d3a700 1 -- 172.21.15.114:0/825041324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 msgr2=0x7f452c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 --2- 172.21.15.114:0/825041324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c0b2090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f452c09b7c0 tx=0x7f452400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 -- 172.21.15.114:0/825041324 shutdown_connections 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 --2- 172.21.15.114:0/825041324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c0b2090 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 --2- 172.21.15.114:0/825041324 >> v2:172.21.15.172:3300/0 conn(0x7f452c0d1b00 0x7f452c0d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 -- 172.21.15.114:0/825041324 >> 172.21.15.114:0/825041324 conn(0x7f452c0730a0 msgr2=0x7f452c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 -- 172.21.15.114:0/825041324 shutdown_connections 2023-11-24T06:25:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.990+0000 7f4530d3a700 1 -- 172.21.15.114:0/825041324 wait complete. 2023-11-24T06:25:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.991+0000 7f4530d3a700 1 Processor -- start 2023-11-24T06:25:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.991+0000 7f4530d3a700 1 -- start start 2023-11-24T06:25:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 0x7f452c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f452c15e110 0x7f452c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15eed0 con 0x7f452c0b1cb0 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15f1b0 con 0x7f452c0d1b00 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4530d3a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c15f490 con 0x7f452c15e110 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40012/0 (socket says 172.21.15.114:40012) 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 -- 172.21.15.114:0/238707775 learned_addr learned my addr 172.21.15.114:0/238707775 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4529d9b700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 0x7f452c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452ad9d700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f452c15e110 0x7f452c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 -- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f452c15e110 msgr2=0x7f452c15bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f452c15e110 0x7f452c15bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 -- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 msgr2=0x7f452c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 0x7f452c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 -- 172.21.15.114:0/238707775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009a70 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f4529d9b700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 0x7f452c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f452a59c700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f452000c550 tx=0x7f452000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.992+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520004c80 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.993+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4520004e00 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.993+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c15f770 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.993+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520007310 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.993+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0cab90 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.994+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c15fdc0 con 0x7f452c0b1cb0 2023-11-24T06:25:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.994+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4520007470 con 0x7f452c0b1cb0 2023-11-24T06:25:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.995+0000 7f451affd700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 0x7f450c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.995+0000 7f4529d9b700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 0x7f450c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.995+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4520089400 con 0x7f452c0b1cb0 2023-11-24T06:25:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.995+0000 7f4529d9b700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 0x7f450c06d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f452c0b75d0 tx=0x7f451c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:28.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:27.999+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4520054390 con 0x7f452c0b1cb0 2023-11-24T06:25:28.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.215+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f452c0cadb0 con 0x7f450c06aed0 2023-11-24T06:25:28.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.217+0000 7f451affd700 1 -- 172.21.15.114:0/238707775 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f452c0cadb0 con 0x7f450c06aed0 2023-11-24T06:25:28.220 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:25:28.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:28.220 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephfs" 2023-11-24T06:25:28.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:28.221 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:25:28.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.218+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 msgr2=0x7f450c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.218+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 0x7f450c06d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f452c0b75d0 tx=0x7f451c008040 comp rx=0 tx=0).stop 2023-11-24T06:25:28.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.218+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 msgr2=0x7f452c15a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.218+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f452000c550 tx=0x7f452000c910 comp rx=0 tx=0).stop 2023-11-24T06:25:28.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 shutdown_connections 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f450c06aed0 0x7f450c06d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f452c15e110 0x7f452c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f452c0b1cb0 0x7f452c15a2d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 --2- 172.21.15.114:0/238707775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c0d1b00 0x7f452c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 >> 172.21.15.114:0/238707775 conn(0x7f452c0730a0 msgr2=0x7f452c0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 shutdown_connections 2023-11-24T06:25:28.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.219+0000 7f4530d3a700 1 -- 172.21.15.114:0/238707775 wait complete. 2023-11-24T06:25:28.234 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:25:28.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 Processor -- start 2023-11-24T06:25:28.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 -- start start 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbcc80afb50 0x7fbcc80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc80ae250 con 0x7fbcc80713b0 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc80af100 con 0x7fbcc80afb50 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.354+0000 7fbccd980700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80ae480 con 0x7fbcc815deb0 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc6ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbcc80afb50 0x7fbcc80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc77fe700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbcc80713b0 0x7fbcc806d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:49328/0 (socket says 172.21.15.114:49328) 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc6ffd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:34564/0 (socket says 172.21.15.114:34564) 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc77fe700 1 -- 172.21.15.114:0/425049512 learned_addr learned my addr 172.21.15.114:0/425049512 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 -- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 msgr2=0x7fbcc80af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 -- 172.21.15.114:0/425049512 >> v1:172.21.15.172:6789/0 conn(0x7fbcc80713b0 legacy=0x7fbcc806d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 -- 172.21.15.114:0/425049512 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80ae760 con 0x7fbcc80afb50 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc5ffb700 1 -- 172.21.15.114:0/425049512 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487334771 0 0) 0x7fbcc80ae250 con 0x7fbcc80713b0 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc6ffd700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:28.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.355+0000 7fbcc67fc700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80afb50 0x7fbcc80b4980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009840 tx=0x7fbcb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25175cb1ad6a7f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:28.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbcc5ffb700 1 -- 172.21.15.114:0/425049512 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0011830 con 0x7fbcc80afb50 2023-11-24T06:25:28.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbcc5ffb700 1 -- 172.21.15.114:0/425049512 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0011990 con 0x7fbcc80afb50 2023-11-24T06:25:28.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbcc5ffb700 1 -- 172.21.15.114:0/425049512 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0011d20 con 0x7fbcc80afb50 2023-11-24T06:25:28.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 -- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80afb50 msgr2=0x7fbcc80b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80afb50 0x7fbcc80b4980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009840 tx=0x7fbcb0009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:28.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 -- 172.21.15.114:0/425049512 shutdown_connections 2023-11-24T06:25:28.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc80af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 --2- 172.21.15.114:0/425049512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80afb50 0x7fbcc80b4980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 -- 172.21.15.114:0/425049512 >> 172.21.15.114:0/425049512 conn(0x7fbcc8074170 msgr2=0x7fbcc806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 -- 172.21.15.114:0/425049512 shutdown_connections 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.356+0000 7fbccd980700 1 -- 172.21.15.114:0/425049512 wait complete. 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.357+0000 7fbccd980700 1 Processor -- start 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 -- start start 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 0x7fbcc8145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81510c0 con 0x7fbcc80afb50 2023-11-24T06:25:28.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81f4b10 con 0x7fbcc80713b0 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbccd980700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81f4830 con 0x7fbcc815deb0 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbcc67fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbcc67fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40028/0 (socket says 172.21.15.114:40028) 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.358+0000 7fbcc67fc700 1 -- 172.21.15.114:0/1997094748 learned_addr learned my addr 172.21.15.114:0/1997094748 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc6ffd700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 0x7fbcc8145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc77fe700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 -- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 msgr2=0x7fbcc815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc815fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 -- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 msgr2=0x7fbcc8145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 0x7fbcc8145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 -- 172.21.15.114:0/1997094748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80094a0 con 0x7fbcc80afb50 2023-11-24T06:25:28.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc6ffd700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 0x7fbcc8145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcc67fc700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00090b0 tx=0x7fbcb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00036a0 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb001f070 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0003a20 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb00094a0 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.359+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc8150ce0 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.360+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc80b3560 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.360+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcb0003b80 con 0x7fbcc80afb50 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.361+0000 7fbcbffff700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 0x7fbca806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.361+0000 7fbcc6ffd700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 0x7fbca806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.368 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.361+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcb0021070 con 0x7fbcc80afb50 2023-11-24T06:25:28.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.361+0000 7fbcc6ffd700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 0x7fbca806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80b08f0 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:28.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.366+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcb0055fd0 con 0x7fbcc80afb50 2023-11-24T06:25:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.557+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbcc80cfb70 con 0x7fbcc80afb50 2023-11-24T06:25:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.557+0000 7fbcbffff700 1 -- 172.21.15.114:0/1997094748 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbcb00150c0 con 0x7fbcc80afb50 2023-11-24T06:25:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 msgr2=0x7fbca806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 0x7fbca806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80b08f0 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 msgr2=0x7fbcc81514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00090b0 tx=0x7fbcb0004750 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 shutdown_connections 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbca806aed0 0x7fbca806d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbcc815deb0 0x7fbcc815fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcc80713b0 0x7fbcc8145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 --2- 172.21.15.114:0/1997094748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcc80afb50 0x7fbcc81514e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 >> 172.21.15.114:0/1997094748 conn(0x7fbcc8074170 msgr2=0x7fbcc806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:28.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 shutdown_connections 2023-11-24T06:25:28.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.559+0000 7fbccd980700 1 -- 172.21.15.114:0/1997094748 wait complete. 2023-11-24T06:25:28.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:28.578 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp2757/subvol9377) 2023-11-24T06:25:28.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:28.614 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-24T06:25:28.671 INFO:teuthology.orchestra.run.smithi114.stdout:cephfs_data 2023-11-24T06:25:28.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:28.672 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp2757/subvol9377) 2023-11-24T06:25:28.752 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:28.752 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp2757/subvol9377/.meta 2023-11-24T06:25:28.824 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol9377 subvol_grp2757 2023-11-24T06:25:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.984+0000 7effbec64700 1 Processor -- start 2023-11-24T06:25:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.985+0000 7effbec64700 1 -- start start 2023-11-24T06:25:28.990 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.985+0000 7effbec64700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 0x7effb806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.985+0000 7effbec64700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80d01c0 con 0x7effb80702e0 2023-11-24T06:25:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.985+0000 7effbec64700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb8070910 con 0x7effb80d21c0 2023-11-24T06:25:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.985+0000 7effbec64700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb8070bf0 con 0x7effb80cf040 2023-11-24T06:25:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb7fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7effb80702e0 0x7effb80d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55744/0 (socket says 172.21.15.114:55744) 2023-11-24T06:25:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb7fff700 1 -- 172.21.15.114:0/3557290806 learned_addr learned my addr 172.21.15.114:0/3557290806 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effbca00700 1 --2- 172.21.15.114:0/3557290806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 0x7effb806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021350742 0 0) 0x7effb80d01c0 con 0x7effb80702e0 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa80035b0 con 0x7effb80702e0 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992571602 0 0) 0x7effb8070910 con 0x7effb80d21c0 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effb80d01c0 con 0x7effb80d21c0 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effbca00700 1 -- 172.21.15.114:0/3557290806 >> v1:172.21.15.172:6789/0 conn(0x7effb80d21c0 legacy=0x7effb80af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.992 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329919048 0 0) 0x7effa80035b0 con 0x7effb80702e0 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effbca00700 1 -- 172.21.15.114:0/3557290806 >> v1:172.21.15.114:6789/0 conn(0x7effb80702e0 legacy=0x7effb80d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effbca00700 1 -- 172.21.15.114:0/3557290806 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80aebb0 con 0x7effb80cf040 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.986+0000 7effbca00700 1 --2- 172.21.15.114:0/3557290806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 0x7effb806fda0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7effac009870 tx=0x7effac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01a604ce6e9ff16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4001670 con 0x7effb80cf040 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac004030 con 0x7effb80cf040 2023-11-24T06:25:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effb77fe700 1 -- 172.21.15.114:0/3557290806 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0043c0 con 0x7effb80cf040 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 -- 172.21.15.114:0/3557290806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 msgr2=0x7effb806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 --2- 172.21.15.114:0/3557290806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 0x7effb806fda0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7effac009870 tx=0x7effac009b80 comp rx=0 tx=0).stop 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 -- 172.21.15.114:0/3557290806 shutdown_connections 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 --2- 172.21.15.114:0/3557290806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80cf040 0x7effb806fda0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 -- 172.21.15.114:0/3557290806 >> 172.21.15.114:0/3557290806 conn(0x7effb809aa00 msgr2=0x7effb809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 -- 172.21.15.114:0/3557290806 shutdown_connections 2023-11-24T06:25:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.987+0000 7effbec64700 1 -- 172.21.15.114:0/3557290806 wait complete. 2023-11-24T06:25:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.988+0000 7effbec64700 1 Processor -- start 2023-11-24T06:25:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.988+0000 7effbec64700 1 -- start start 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 0x7effb80cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 0x7effb80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81f3100 con 0x7effb80cf040 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb815bc90 con 0x7effb80d21c0 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effbec64700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb815ad90 con 0x7effb80702e0 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effb7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effb7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40042/0 (socket says 172.21.15.114:40042) 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.989+0000 7effb7fff700 1 -- 172.21.15.114:0/2786147205 learned_addr learned my addr 172.21.15.114:0/2786147205 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbca00700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 0x7effb80cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbd201700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 0x7effb80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 -- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 msgr2=0x7effb80cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 0x7effb80cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 -- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 msgr2=0x7effb80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 0x7effb80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 -- 172.21.15.114:0/2786147205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac0094a0 con 0x7effb80cf040 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbca00700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 0x7effb80cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbd201700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 0x7effb80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb7fff700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7effa400c750 tx=0x7effa400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4007560 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa40076c0 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4004ab0 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb815b3b0 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.990+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb815a460 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.991+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb8067f60 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.992+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effa4004c10 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.992+0000 7effb57fa700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 0x7eff9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.992+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effa4088650 con 0x7effb80cf040 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.992+0000 7effbca00700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 0x7eff9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.992+0000 7effbca00700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 0x7eff9c06d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effac0097c0 tx=0x7effac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:28.996+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effa4053790 con 0x7effb80cf040 2023-11-24T06:25:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.220+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol9377", "group_name": "subvol_grp2757", "target": ["mon-mgr", ""]}) v1 -- 0x7effb80690c0 con 0x7eff9c06aed0 2023-11-24T06:25:29.228 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:29.225+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:29.229 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:29.225+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:29.229 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:29.225+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:29.229 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:29.225+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:29.229 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:29.225+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:29.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.262+0000 7effb57fa700 1 -- 172.21.15.114:0/2786147205 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7effb80690c0 con 0x7eff9c06aed0 2023-11-24T06:25:29.265 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/subvol_grp2757/subvol9377 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 msgr2=0x7eff9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 0x7eff9c06d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effac0097c0 tx=0x7effac008030 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 msgr2=0x7effb815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7effa400c750 tx=0x7effa400cb10 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 shutdown_connections 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff9c06aed0 0x7eff9c06d390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb80d21c0 0x7effb80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7effb80702e0 0x7effb80cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 --2- 172.21.15.114:0/2786147205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7effb80cf040 0x7effb815b750 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 >> 172.21.15.114:0/2786147205 conn(0x7effb809aa00 msgr2=0x7effb80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:29.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 shutdown_connections 2023-11-24T06:25:29.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.264+0000 7effbec64700 1 -- 172.21.15.114:0/2786147205 wait complete. 2023-11-24T06:25:29.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol9377 subvol_grp2757 2023-11-24T06:25:29.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.400+0000 7f1c64f13700 1 Processor -- start 2023-11-24T06:25:29.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.400+0000 7f1c64f13700 1 -- start start 2023-11-24T06:25:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c64f13700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c600746a0 0x7f1c6006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c64f13700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c64f13700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c600b5e50 con 0x7f1c600b01e0 2023-11-24T06:25:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c64f13700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c601470a0 con 0x7f1c600746a0 2023-11-24T06:25:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c64f13700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600b05c0 con 0x7f1c600b59c0 2023-11-24T06:25:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c5ed9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c5dd9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c600746a0 0x7f1c6006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c5ed9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:34614/0 (socket says 172.21.15.114:34614) 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c5ed9d700 1 -- 172.21.15.114:0/1523473913 learned_addr learned my addr 172.21.15.114:0/1523473913 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.401+0000 7f1c5e59c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1c600b01e0 0x7f1c60073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:49370/0 (socket says 172.21.15.114:49370) 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5dd9b700 1 -- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 msgr2=0x7f1c600b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5dd9b700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5dd9b700 1 -- 172.21.15.114:0/1523473913 >> v1:172.21.15.172:6789/0 conn(0x7f1c600b01e0 legacy=0x7f1c60073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5d59a700 1 -- 172.21.15.114:0/1523473913 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260008229 0 0) 0x7f1c54001110 con 0x7f1c600b01e0 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5dd9b700 1 -- 172.21.15.114:0/1523473913 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600b4250 con 0x7f1c600746a0 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5ed9d700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5dd9b700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600746a0 0x7f1c6006e9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009840 tx=0x7f1c50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d2f10118509501 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5d59a700 1 -- 172.21.15.114:0/1523473913 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50011830 con 0x7f1c600746a0 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5d59a700 1 -- 172.21.15.114:0/1523473913 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50011990 con 0x7f1c600746a0 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.402+0000 7f1c5d59a700 1 -- 172.21.15.114:0/1523473913 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50011d20 con 0x7f1c600746a0 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 -- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600746a0 msgr2=0x7f1c6006e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600746a0 0x7f1c6006e9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009840 tx=0x7f1c50009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 -- 172.21.15.114:0/1523473913 shutdown_connections 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600b59c0 0x7f1c600b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 --2- 172.21.15.114:0/1523473913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600746a0 0x7f1c6006e9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 -- 172.21.15.114:0/1523473913 >> 172.21.15.114:0/1523473913 conn(0x7f1c600d0b40 msgr2=0x7f1c600d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 -- 172.21.15.114:0/1523473913 shutdown_connections 2023-11-24T06:25:29.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.403+0000 7f1c64f13700 1 -- 172.21.15.114:0/1523473913 wait complete. 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.404+0000 7f1c64f13700 1 Processor -- start 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.404+0000 7f1c64f13700 1 -- start start 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600746a0 0x7f1c600cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 0x7f1c6015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6015d870 con 0x7f1c600b59c0 2023-11-24T06:25:29.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60159c30 con 0x7f1c600b01e0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60159f10 con 0x7f1c600746a0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40076/0 (socket says 172.21.15.114:40076) 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 -- 172.21.15.114:0/3337217913 learned_addr learned my addr 172.21.15.114:0/3337217913 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5dd9b700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 0x7f1c6015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5e59c700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600746a0 0x7f1c600cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 -- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600746a0 msgr2=0x7f1c600cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600746a0 0x7f1c600cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 -- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 msgr2=0x7f1c6015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 0x7f1c6015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 -- 172.21.15.114:0/3337217913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009b30 con 0x7f1c600b59c0 2023-11-24T06:25:29.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5dd9b700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 0x7f1c6015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c5ed9d700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800efc0 tx=0x7f1c58009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58018070 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800f040 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.405+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c500094a0 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.406+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58003c40 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.406+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60159370 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.406+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c60068670 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.407+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c5800c9e0 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.407+0000 7f1c4effd700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 0x7f1c4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.407+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c58089880 con 0x7f1c600b59c0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.407+0000 7f1c5e59c700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 0x7f1c4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.408+0000 7f1c5e59c700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 0x7f1c4006d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000f60 tx=0x7f1c54007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.412+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c580549c0 con 0x7f1c600b59c0 2023-11-24T06:25:29.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.635+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol9377", "group_name": "subvol_grp2757", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c60159da0 con 0x7f1c4006aed0 2023-11-24T06:25:29.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.639+0000 7f1c4effd700 1 -- 172.21.15.114:0/3337217913 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7f1c60159da0 con 0x7f1c4006aed0 2023-11-24T06:25:29.643 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/subvol_grp2757/subvol9377 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 msgr2=0x7f1c4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 0x7f1c4006d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000f60 tx=0x7f1c54007a40 comp rx=0 tx=0).stop 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 msgr2=0x7f1c60142610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800efc0 tx=0x7f1c58009f90 comp rx=0 tx=0).stop 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 shutdown_connections 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c4006aed0 0x7f1c4006d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c600b59c0 0x7f1c60142610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c600746a0 0x7f1c600cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 --2- 172.21.15.114:0/3337217913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c600b01e0 0x7f1c6015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 >> 172.21.15.114:0/3337217913 conn(0x7f1c600d0b40 msgr2=0x7f1c60072450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:29.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.641+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 shutdown_connections 2023-11-24T06:25:29.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.642+0000 7f1c64f13700 1 -- 172.21.15.114:0/3337217913 wait complete. 2023-11-24T06:25:29.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:29.657 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/2305c23e58e701d3bd86d1f4adf4944b.meta) 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout:[GLOBAL] 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout:version = 1 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout:type = subvolume 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout:path = /volumes/subvol_grp2757/subvol9377 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout:state = complete 2023-11-24T06:25:29.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:29.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol9377 subvol_grp2757 2023-11-24T06:25:29.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 Processor -- start 2023-11-24T06:25:29.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 -- start start 2023-11-24T06:25:29.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a6815de80 0x7f0a680cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a680d1b00 0x7f0a680d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a680d2450 con 0x7f0a6815e260 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6806b580 con 0x7f0a6815de80 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.849+0000 7f0a6e0a3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680d02d0 con 0x7f0a680d1b00 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a6815de80 0x7f0a680cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a6815de80 0x7f0a680cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40082/0 (socket says 172.21.15.114:40082) 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 -- 172.21.15.114:0/3662378095 learned_addr learned my addr 172.21.15.114:0/3662378095 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a67fff700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a680d1b00 0x7f0a680d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 -- 172.21.15.114:0/3662378095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a680d1b00 msgr2=0x7f0a680d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a680d1b00 0x7f0a680d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 -- 172.21.15.114:0/3662378095 >> v1:172.21.15.172:6789/0 conn(0x7f0a6815e260 legacy=0x7f0a6806ed70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 -- 172.21.15.114:0/3662378095 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6806f810 con 0x7f0a6815de80 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a677fe700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815de80 0x7f0a680cfd90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098f0 tx=0x7f0a58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813aa7135e755497 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a66ffd700 1 -- 172.21.15.114:0/3662378095 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004230 con 0x7f0a6815de80 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a66ffd700 1 -- 172.21.15.114:0/3662378095 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a58004390 con 0x7f0a6815de80 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.850+0000 7f0a66ffd700 1 -- 172.21.15.114:0/3662378095 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580036a0 con 0x7f0a6815de80 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/3662378095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815de80 msgr2=0x7f0a680cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815de80 0x7f0a680cfd90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098f0 tx=0x7f0a58009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/3662378095 shutdown_connections 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a680d1b00 0x7f0a680d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/3662378095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815de80 0x7f0a680cfd90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/3662378095 >> 172.21.15.114:0/3662378095 conn(0x7f0a6809aa00 msgr2=0x7f0a6809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:29.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/3662378095 shutdown_connections 2023-11-24T06:25:29.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.851+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/3662378095 wait complete. 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.852+0000 7f0a6e0a3700 1 Processor -- start 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.852+0000 7f0a6e0a3700 1 -- start start 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a680d1b00 0x7f0a68150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6815de80 0x7f0a680afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680b40d0 con 0x7f0a6815e260 2023-11-24T06:25:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681f4a60 con 0x7f0a680d1b00 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.853+0000 7f0a6e0a3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681f5040 con 0x7f0a6815de80 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40092/0 (socket says 172.21.15.114:40092) 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 -- 172.21.15.114:0/2589201800 learned_addr learned my addr 172.21.15.114:0/2589201800 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a677fe700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a680d1b00 0x7f0a68150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 -- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6815de80 msgr2=0x7f0a680afa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6815de80 0x7f0a680afa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 -- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a680d1b00 msgr2=0x7f0a68150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a680d1b00 0x7f0a68150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 -- 172.21.15.114:0/2589201800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a58009550 con 0x7f0a6815e260 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a67fff700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000efc0 tx=0x7f0a6000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000cb80 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6000f040 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60014480 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a681f5320 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.854+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a681f4d40 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.855+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a681f5970 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.855+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a600145e0 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.856+0000 7f0a64ff9700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 0x7f0a4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.856+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a60089da0 con 0x7f0a6815e260 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.856+0000 7f0a677fe700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 0x7f0a4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.856+0000 7f0a677fe700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 0x7f0a4806d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009130 tx=0x7f0a58008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:29.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:29.860+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a60054ee0 con 0x7f0a6815e260 2023-11-24T06:25:30.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.080+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol9377", "group_name": "subvol_grp2757", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a680700b0 con 0x7f0a4806aed0 2023-11-24T06:25:30.096 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.093+0000 7f0a64ff9700 1 -- 172.21.15.114:0/2589201800 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0a58006c80 con 0x7f0a4806aed0 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 msgr2=0x7f0a4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 0x7f0a4806d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009130 tx=0x7f0a58008030 comp rx=0 tx=0).stop 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 msgr2=0x7f0a680aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000efc0 tx=0x7f0a6000c0c0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 shutdown_connections 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a4806aed0 0x7f0a4806d390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a6815e260 0x7f0a680aff80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a680d1b00 0x7f0a68150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 --2- 172.21.15.114:0/2589201800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6815de80 0x7f0a680afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 >> 172.21.15.114:0/2589201800 conn(0x7f0a6809aa00 msgr2=0x7f0a680cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:30.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 shutdown_connections 2023-11-24T06:25:30.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.095+0000 7f0a6e0a3700 1 -- 172.21.15.114:0/2589201800 wait complete. 2023-11-24T06:25:30.100 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.096+0000 7f607c8e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.100 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.096+0000 7f607c8e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.100 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.096+0000 7f607c8e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.101 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.096+0000 7f607c8e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.101 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.096+0000 7f607c8e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:30.114 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:25:30.114 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.110+0000 7f607b0df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.114 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.110+0000 7f607b0df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.114 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.110+0000 7f607b0df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.114 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.110+0000 7f607b0df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.115 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:25:30.110+0000 7f607b0df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:25:30.147 INFO:teuthology.orchestra.run.smithi114.stdout:2 2023-11-24T06:25:30.148 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp2757 2023-11-24T06:25:30.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.309+0000 7f5a25e9b700 1 Processor -- start 2023-11-24T06:25:30.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.309+0000 7f5a25e9b700 1 -- start start 2023-11-24T06:25:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a25e9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a200c6c30 0x7f5a200cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a25e9b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200d0140 con 0x7f5a200cef40 2023-11-24T06:25:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a25e9b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200cf3d0 con 0x7f5a2015bf50 2023-11-24T06:25:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a25e9b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2015c330 con 0x7f5a200c6c30 2023-11-24T06:25:30.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a1ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5a200cef40 0x7f5a200cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55836/0 (socket says 172.21.15.114:55836) 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a17fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5a200c6c30 0x7f5a200cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a1ffff700 1 -- 172.21.15.114:0/611238270 learned_addr learned my addr 172.21.15.114:0/611238270 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a17fff700 1 -- 172.21.15.114:0/611238270 >> v1:172.21.15.172:6789/0 conn(0x7f5a2015bf50 legacy=0x7f5a200ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a17fff700 1 -- 172.21.15.114:0/611238270 >> v1:172.21.15.114:6789/0 conn(0x7f5a200cef40 legacy=0x7f5a200cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a1effd700 1 -- 172.21.15.114:0/611238270 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712491118 0 0) 0x7f5a200d0140 con 0x7f5a200cef40 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.310+0000 7f5a17fff700 1 -- 172.21.15.114:0/611238270 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200d03e0 con 0x7f5a200c6c30 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a17fff700 1 --2- 172.21.15.114:0/611238270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200c6c30 0x7f5a200cea00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009840 tx=0x7f5a08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db45f1522cd45e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a1effd700 1 -- 172.21.15.114:0/611238270 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08011830 con 0x7f5a200c6c30 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a1effd700 1 -- 172.21.15.114:0/611238270 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08011990 con 0x7f5a200c6c30 2023-11-24T06:25:30.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a1effd700 1 -- 172.21.15.114:0/611238270 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08011d20 con 0x7f5a200c6c30 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 -- 172.21.15.114:0/611238270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200c6c30 msgr2=0x7f5a200cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/611238270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200c6c30 0x7f5a200cea00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009840 tx=0x7f5a08009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 -- 172.21.15.114:0/611238270 shutdown_connections 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/611238270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200c6c30 0x7f5a200cea00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 -- 172.21.15.114:0/611238270 >> 172.21.15.114:0/611238270 conn(0x7f5a200b3780 msgr2=0x7f5a200b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 -- 172.21.15.114:0/611238270 shutdown_connections 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.311+0000 7f5a25e9b700 1 -- 172.21.15.114:0/611238270 wait complete. 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 Processor -- start 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 -- start start 2023-11-24T06:25:30.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a200c6c30 0x7f5a2015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 0x7f5a2006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200c9020 con 0x7f5a200cef40 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2006b090 con 0x7f5a2015bf50 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a25e9b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2006b370 con 0x7f5a200c6c30 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40132/0 (socket says 172.21.15.114:40132) 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a17fff700 1 -- 172.21.15.114:0/1352111906 learned_addr learned my addr 172.21.15.114:0/1352111906 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.313+0000 7f5a1ffff700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 0x7f5a2006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a1f7fe700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a200c6c30 0x7f5a2015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 -- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a200c6c30 msgr2=0x7f5a2015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a200c6c30 0x7f5a2015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 -- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 msgr2=0x7f5a2006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 0x7f5a2006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 -- 172.21.15.114:0/1352111906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080094a0 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a1ffff700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 0x7f5a2006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a17fff700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5a080090b0 tx=0x7f5a08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0801a900 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0801aa60 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2006b650 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0801add0 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.314+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2006bc10 con 0x7f5a200cef40 2023-11-24T06:25:30.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.315+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2006bda0 con 0x7f5a200cef40 2023-11-24T06:25:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.315+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a08022070 con 0x7f5a200cef40 2023-11-24T06:25:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.316+0000 7f5a1cff9700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 0x7f5a0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.316+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a0808a820 con 0x7f5a200cef40 2023-11-24T06:25:30.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.316+0000 7f5a1f7fe700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 0x7f5a0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.316+0000 7f5a1f7fe700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 0x7f5a0006d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000f60 tx=0x7f5a10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.320+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a08055a10 con 0x7f5a200cef40 2023-11-24T06:25:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.545+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp2757", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a200c5700 con 0x7f5a0006aed0 2023-11-24T06:25:30.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.549+0000 7f5a1cff9700 1 -- 172.21.15.114:0/1352111906 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5a10001340 con 0x7f5a0006aed0 2023-11-24T06:25:30.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 msgr2=0x7f5a0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 0x7f5a0006d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a10000f60 tx=0x7f5a10009040 comp rx=0 tx=0).stop 2023-11-24T06:25:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 msgr2=0x7f5a20148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5a080090b0 tx=0x7f5a08004750 comp rx=0 tx=0).stop 2023-11-24T06:25:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 shutdown_connections 2023-11-24T06:25:30.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5a0006aed0 0x7f5a0006d390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a2015bf50 0x7f5a2006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a200c6c30 0x7f5a2015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 --2- 172.21.15.114:0/1352111906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a200cef40 0x7f5a20148490 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 >> 172.21.15.114:0/1352111906 conn(0x7f5a200b3780 msgr2=0x7f5a200c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 shutdown_connections 2023-11-24T06:25:30.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.551+0000 7f5a25e9b700 1 -- 172.21.15.114:0/1352111906 wait complete. 2023-11-24T06:25:30.566 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:25:30.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:25:30.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:25:30.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:25:30.657 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:30.657 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:30.674 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:30.675 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:30.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.893+0000 7ff9d4d5a700 1 Processor -- start 2023-11-24T06:25:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.893+0000 7ff9d4d5a700 1 -- start start 2023-11-24T06:25:30.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.894+0000 7ff9d4d5a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff9d0152cc0 0x7ff9d006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.894+0000 7ff9d4d5a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff9d00b49a0 0x7ff9d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.894+0000 7ff9d4d5a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d015c4b0 con 0x7ff9d00cff20 2023-11-24T06:25:30.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.894+0000 7ff9d4d5a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d015c790 con 0x7ff9d0152cc0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.894+0000 7ff9d4d5a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d015ca70 con 0x7ff9d00b49a0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff9d0152cc0 0x7ff9d006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ced9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff9d00cff20 0x7ff9d00d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55850/0 (socket says 172.21.15.114:55850) 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ced9d700 1 -- 172.21.15.114:0/4206294903 learned_addr learned my addr 172.21.15.114:0/4206294903 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9cdd9b700 1 --2- 172.21.15.114:0/4206294903 >> v2:172.21.15.172:3300/0 conn(0x7ff9d00b49a0 0x7ff9d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 -- 172.21.15.114:0/4206294903 >> v2:172.21.15.172:3300/0 conn(0x7ff9d00b49a0 msgr2=0x7ff9d006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 --2- 172.21.15.114:0/4206294903 >> v2:172.21.15.172:3300/0 conn(0x7ff9d00b49a0 0x7ff9d006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 -- 172.21.15.114:0/4206294903 >> v1:172.21.15.114:6789/0 conn(0x7ff9d00cff20 legacy=0x7ff9d00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 -- 172.21.15.114:0/4206294903 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d015cd50 con 0x7ff9d0152cc0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9cd59a700 1 -- 172.21.15.114:0/4206294903 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096369619 0 0) 0x7ff9d015c4b0 con 0x7ff9d00cff20 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9ce59c700 1 --2- 172.21.15.114:0/4206294903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d0152cc0 0x7ff9d006c000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b4e50 tx=0x7ff9c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3414ead54237daf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9cd59a700 1 -- 172.21.15.114:0/4206294903 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4011830 con 0x7ff9d0152cc0 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9cd59a700 1 -- 172.21.15.114:0/4206294903 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4011990 con 0x7ff9d0152cc0 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.895+0000 7ff9cd59a700 1 -- 172.21.15.114:0/4206294903 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4011d20 con 0x7ff9d0152cc0 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/4206294903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d0152cc0 msgr2=0x7ff9d006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/4206294903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d0152cc0 0x7ff9d006c000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b4e50 tx=0x7ff9c4009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/4206294903 shutdown_connections 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/4206294903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d0152cc0 0x7ff9d006c000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/4206294903 >> v2:172.21.15.172:3300/0 conn(0x7ff9d00b49a0 0x7ff9d006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/4206294903 >> 172.21.15.114:0/4206294903 conn(0x7ff9d0071b90 msgr2=0x7ff9d0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/4206294903 shutdown_connections 2023-11-24T06:25:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.896+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/4206294903 wait complete. 2023-11-24T06:25:30.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 Processor -- start 2023-11-24T06:25:30.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 -- start start 2023-11-24T06:25:30.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9d00b49a0 0x7ff9d01425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d0152cc0 0x7ff9d00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0154e80 con 0x7ff9d00cff20 2023-11-24T06:25:30.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0142ae0 con 0x7ff9d0152cc0 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.898+0000 7ff9d4d5a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0157fa0 con 0x7ff9d00b49a0 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40146/0 (socket says 172.21.15.114:40146) 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 -- 172.21.15.114:0/2495000339 learned_addr learned my addr 172.21.15.114:0/2495000339 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9ced9d700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d0152cc0 0x7ff9d00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9ce59c700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9d00b49a0 0x7ff9d01425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 -- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9d00b49a0 msgr2=0x7ff9d01425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9d00b49a0 0x7ff9d01425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 -- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d0152cc0 msgr2=0x7ff9d00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d0152cc0 0x7ff9d00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 -- 172.21.15.114:0/2495000339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40094a0 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9cdd9b700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9b0 tx=0x7ff9c000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0009710 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c000ee50 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d0154830 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000ca50 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.899+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d0158280 con 0x7ff9d00cff20 2023-11-24T06:25:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.900+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d00753f0 con 0x7ff9d00cff20 2023-11-24T06:25:30.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.900+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff9c000cbb0 con 0x7ff9d00cff20 2023-11-24T06:25:30.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.901+0000 7ff9beffd700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 0x7ff9b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:30.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.901+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9c0089c40 con 0x7ff9d00cff20 2023-11-24T06:25:30.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.901+0000 7ff9ce59c700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 0x7ff9b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:30.908 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.901+0000 7ff9ce59c700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 0x7ff9b006d540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40090b0 tx=0x7ff9c401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:30.909 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:30.905+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9c0054a70 con 0x7ff9d00cff20 2023-11-24T06:25:31.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.099+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9d0067f60 con 0x7ff9d00cff20 2023-11-24T06:25:31.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.100+0000 7ff9beffd700 1 -- 172.21.15.114:0/2495000339 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7ff9c0017090 con 0x7ff9d00cff20 2023-11-24T06:25:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:31.104 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:31.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 msgr2=0x7ff9b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 0x7ff9b006d540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40090b0 tx=0x7ff9c401c040 comp rx=0 tx=0).stop 2023-11-24T06:25:31.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 msgr2=0x7ff9d0157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000e9b0 tx=0x7ff9c000ecc0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 shutdown_connections 2023-11-24T06:25:31.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff9b006b080 0x7ff9b006d540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d0152cc0 0x7ff9d00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9d00b49a0 0x7ff9d01425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 --2- 172.21.15.114:0/2495000339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9d00cff20 0x7ff9d0157750 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 >> 172.21.15.114:0/2495000339 conn(0x7ff9d0071b90 msgr2=0x7ff9d0073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 shutdown_connections 2023-11-24T06:25:31.108 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.102+0000 7ff9d4d5a700 1 -- 172.21.15.114:0/2495000339 wait complete. 2023-11-24T06:25:31.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:15.659348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:16.837675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:31.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.237+0000 7fd4d3726700 1 Processor -- start 2023-11-24T06:25:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 -- start start 2023-11-24T06:25:31.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4cc1587f0 0x7fd4cc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4cc0b4370 0x7fd4cc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc06b530 con 0x7fd4cc15b370 2023-11-24T06:25:31.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0ae5e0 con 0x7fd4cc0b4370 2023-11-24T06:25:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d3726700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc15be30 con 0x7fd4cc1587f0 2023-11-24T06:25:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d1cc3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4cc0b4370 0x7fd4cc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d0cc1700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd4cc15b370 0x7fd4cc0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55860/0 (socket says 172.21.15.114:55860) 2023-11-24T06:25:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.238+0000 7fd4d0cc1700 1 -- 172.21.15.114:0/1646803919 learned_addr learned my addr 172.21.15.114:0/1646803919 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d14c2700 1 --2- 172.21.15.114:0/1646803919 >> v2:172.21.15.172:3300/0 conn(0x7fd4cc1587f0 0x7fd4cc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/1646803919 >> v2:172.21.15.172:3300/0 conn(0x7fd4cc1587f0 msgr2=0x7fd4cc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d1cc3700 1 --2- 172.21.15.114:0/1646803919 >> v2:172.21.15.172:3300/0 conn(0x7fd4cc1587f0 0x7fd4cc06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/1646803919 >> v1:172.21.15.114:6789/0 conn(0x7fd4cc15b370 legacy=0x7fd4cc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4c3fff700 1 -- 172.21.15.114:0/1646803919 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860626289 0 0) 0x7fd4cc06b530 con 0x7fd4cc15b370 2023-11-24T06:25:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/1646803919 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc15c110 con 0x7fd4cc0b4370 2023-11-24T06:25:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4d1cc3700 1 --2- 172.21.15.114:0/1646803919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc15b800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009d90 tx=0x7fd4c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d940f7910bc828 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4c3fff700 1 -- 172.21.15.114:0/1646803919 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400e070 con 0x7fd4cc0b4370 2023-11-24T06:25:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4c3fff700 1 -- 172.21.15.114:0/1646803919 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c400bb90 con 0x7fd4cc0b4370 2023-11-24T06:25:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.239+0000 7fd4c3fff700 1 -- 172.21.15.114:0/1646803919 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4012430 con 0x7fd4cc0b4370 2023-11-24T06:25:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 -- 172.21.15.114:0/1646803919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc0b4370 msgr2=0x7fd4cc15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 --2- 172.21.15.114:0/1646803919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc15b800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009d90 tx=0x7fd4c400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 -- 172.21.15.114:0/1646803919 shutdown_connections 2023-11-24T06:25:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 --2- 172.21.15.114:0/1646803919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc15b800 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 --2- 172.21.15.114:0/1646803919 >> v2:172.21.15.172:3300/0 conn(0x7fd4cc1587f0 0x7fd4cc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 -- 172.21.15.114:0/1646803919 >> 172.21.15.114:0/1646803919 conn(0x7fd4cc0d0bf0 msgr2=0x7fd4cc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 -- 172.21.15.114:0/1646803919 shutdown_connections 2023-11-24T06:25:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.240+0000 7fd4d3726700 1 -- 172.21.15.114:0/1646803919 wait complete. 2023-11-24T06:25:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.241+0000 7fd4d3726700 1 Processor -- start 2023-11-24T06:25:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.241+0000 7fd4d3726700 1 -- start start 2023-11-24T06:25:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4cc1587f0 0x7fd4cc14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc15ace0 con 0x7fd4cc15b370 2023-11-24T06:25:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc14d5d0 con 0x7fd4cc0b4370 2023-11-24T06:25:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc15a060 con 0x7fd4cc1587f0 2023-11-24T06:25:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40162/0 (socket says 172.21.15.114:40162) 2023-11-24T06:25:31.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/2736767052 learned_addr learned my addr 172.21.15.114:0/2736767052 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:31.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d14c2700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d0cc1700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4cc1587f0 0x7fd4cc14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4cc1587f0 msgr2=0x7fd4cc14d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4cc1587f0 0x7fd4cc14d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4cc0b4370 msgr2=0x7fd4cc159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 -- 172.21.15.114:0/2736767052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c4009a70 con 0x7fd4cc15b370 2023-11-24T06:25:31.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d1cc3700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0d1730 tx=0x7fd4c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4012430 con 0x7fd4cc15b370 2023-11-24T06:25:31.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4020070 con 0x7fd4cc15b370 2023-11-24T06:25:31.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc15a340 con 0x7fd4cc15b370 2023-11-24T06:25:31.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.242+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004280 con 0x7fd4cc15b370 2023-11-24T06:25:31.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.243+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc15a900 con 0x7fd4cc15b370 2023-11-24T06:25:31.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.243+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc068fb0 con 0x7fd4cc15b370 2023-11-24T06:25:31.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.244+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd4c40043e0 con 0x7fd4cc15b370 2023-11-24T06:25:31.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.245+0000 7fd4c1ffb700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 0x7fd4b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.245+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4c4022070 con 0x7fd4cc15b370 2023-11-24T06:25:31.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.245+0000 7fd4d14c2700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 0x7fd4b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.245+0000 7fd4d14c2700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 0x7fd4b006d540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006b20 tx=0x7fd4bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.248+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4c4056e80 con 0x7fd4cc15b370 2023-11-24T06:25:31.440 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.437+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4cc068270 con 0x7fd4cc15b370 2023-11-24T06:25:31.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.437+0000 7fd4c1ffb700 1 -- 172.21.15.114:0/2736767052 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4c405a4a0 con 0x7fd4cc15b370 2023-11-24T06:25:31.441 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:31.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 msgr2=0x7fd4b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 0x7fd4b006d540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006b20 tx=0x7fd4bc008040 comp rx=0 tx=0).stop 2023-11-24T06:25:31.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 msgr2=0x7fd4cc1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0d1730 tx=0x7fd4c4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 shutdown_connections 2023-11-24T06:25:31.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd4b006b080 0x7fd4b006d540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4cc15b370 0x7fd4cc1445b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4cc0b4370 0x7fd4cc159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 --2- 172.21.15.114:0/2736767052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4cc1587f0 0x7fd4cc14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 >> 172.21.15.114:0/2736767052 conn(0x7fd4cc0d0bf0 msgr2=0x7fd4cc06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 shutdown_connections 2023-11-24T06:25:31.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.440+0000 7fd4d3726700 1 -- 172.21.15.114:0/2736767052 wait complete. 2023-11-24T06:25:31.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.576+0000 7f49c82f4700 1 Processor -- start 2023-11-24T06:25:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.576+0000 7f49c82f4700 1 -- start start 2023-11-24T06:25:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.577+0000 7f49c82f4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f49c00b5da0 0x7f49c00afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.577+0000 7f49c82f4700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b3c60 con 0x7f49c015e160 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.577+0000 7f49c82f4700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b58c0 con 0x7f49c00746a0 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.577+0000 7f49c82f4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00b4500 con 0x7f49c00b5da0 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6090700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f49c015e160 0x7f49c0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55874/0 (socket says 172.21.15.114:55874) 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6090700 1 -- 172.21.15.114:0/1519371314 learned_addr learned my addr 172.21.15.114:0/1519371314 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6891700 1 --2- 172.21.15.114:0/1519371314 >> v2:172.21.15.114:3300/0 conn(0x7f49c00b5da0 0x7f49c00afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c508e700 1 -- 172.21.15.114:0/1519371314 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084116636 0 0) 0x7f49c00b3c60 con 0x7f49c015e160 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c508e700 1 -- 172.21.15.114:0/1519371314 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac0035b0 con 0x7f49c015e160 2023-11-24T06:25:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6891700 1 -- 172.21.15.114:0/1519371314 >> v1:172.21.15.172:6789/0 conn(0x7f49c00746a0 legacy=0x7f49c006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6891700 1 -- 172.21.15.114:0/1519371314 >> v1:172.21.15.114:6789/0 conn(0x7f49c015e160 legacy=0x7f49c0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6891700 1 -- 172.21.15.114:0/1519371314 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c006bdc0 con 0x7f49c00b5da0 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c6891700 1 --2- 172.21.15.114:0/1519371314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c00b5da0 0x7f49c00afc90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009d90 tx=0x7f49b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0a40352fd5718c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c508e700 1 -- 172.21.15.114:0/1519371314 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800e070 con 0x7f49c00b5da0 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c508e700 1 -- 172.21.15.114:0/1519371314 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b800bb20 con 0x7f49c00b5da0 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.578+0000 7f49c508e700 1 -- 172.21.15.114:0/1519371314 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8012430 con 0x7f49c00b5da0 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 -- 172.21.15.114:0/1519371314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c00b5da0 msgr2=0x7f49c00afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 --2- 172.21.15.114:0/1519371314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c00b5da0 0x7f49c00afc90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009d90 tx=0x7f49b800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 -- 172.21.15.114:0/1519371314 shutdown_connections 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 --2- 172.21.15.114:0/1519371314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c00b5da0 0x7f49c00afc90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 -- 172.21.15.114:0/1519371314 >> 172.21.15.114:0/1519371314 conn(0x7f49c00d0bf0 msgr2=0x7f49c00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 -- 172.21.15.114:0/1519371314 shutdown_connections 2023-11-24T06:25:31.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.579+0000 7f49c82f4700 1 -- 172.21.15.114:0/1519371314 wait complete. 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.580+0000 7f49c82f4700 1 Processor -- start 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.580+0000 7f49c82f4700 1 -- start start 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00746a0 0x7f49c00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49c00b5da0 0x7f49c014fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0153d60 con 0x7f49c015e160 2023-11-24T06:25:31.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01604d0 con 0x7f49c00746a0 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b09e0 con 0x7f49c00b5da0 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40176/0 (socket says 172.21.15.114:40176) 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 -- 172.21.15.114:0/3656387047 learned_addr learned my addr 172.21.15.114:0/3656387047 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6090700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00746a0 0x7f49c00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 -- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49c00b5da0 msgr2=0x7f49c014fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49c00b5da0 0x7f49c014fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 -- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00746a0 msgr2=0x7f49c00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00746a0 0x7f49c00b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 -- 172.21.15.114:0/3656387047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009a70 con 0x7f49c015e160 2023-11-24T06:25:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c6891700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f49c0072ed0 tx=0x7f49b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8012430 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8020070 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004280 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.581+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00b0cc0 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.582+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00b1280 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.582+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c0153ac0 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.583+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f49b80043e0 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.583+0000 7f49b6ffd700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 0x7f49a406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.583+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49b8022070 con 0x7f49c015e160 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.583+0000 7f49c6090700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 0x7f49a406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.584+0000 7f49c6090700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 0x7f49a406d540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49b00011e0 tx=0x7f49b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.588+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49c0153ac0 con 0x7f49c015e160 2023-11-24T06:25:31.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.779+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49c0153ac0 con 0x7f49c015e160 2023-11-24T06:25:31.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.779+0000 7f49b6ffd700 1 -- 172.21.15.114:0/3656387047 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49c0153ac0 con 0x7f49c015e160 2023-11-24T06:25:31.783 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:31.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 msgr2=0x7f49a406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 0x7f49a406d540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49b00011e0 tx=0x7f49b0009040 comp rx=0 tx=0).stop 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 msgr2=0x7f49c00b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f49c0072ed0 tx=0x7f49b8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 shutdown_connections 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f49a406b080 0x7f49a406d540 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49c015e160 0x7f49c00b0350 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49c00746a0 0x7f49c00b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 --2- 172.21.15.114:0/3656387047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49c00b5da0 0x7f49c014fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 >> 172.21.15.114:0/3656387047 conn(0x7f49c00d0bf0 msgr2=0x7f49c006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 shutdown_connections 2023-11-24T06:25:31.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.781+0000 7f49c82f4700 1 -- 172.21.15.114:0/3656387047 wait complete. 2023-11-24T06:25:31.796 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:31.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 Processor -- start 2023-11-24T06:25:31.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 -- start start 2023-11-24T06:25:31.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e4015bf50 0x7f6e400ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400d0140 con 0x7f6e400ce830 2023-11-24T06:25:31.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e400cf3c0 con 0x7f6e400c6c30 2023-11-24T06:25:31.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e44fe7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4015c330 con 0x7f6e4015bf50 2023-11-24T06:25:31.931 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3dd9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6e400ce830 0x7f6e400cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55890/0 (socket says 172.21.15.114:55890) 2023-11-24T06:25:31.932 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3dd9b700 1 -- 172.21.15.114:0/1022283802 learned_addr learned my addr 172.21.15.114:0/1022283802 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:31.932 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3e59c700 1 --2- 172.21.15.114:0/1022283802 >> v2:172.21.15.172:3300/0 conn(0x7f6e4015bf50 0x7f6e400ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.932 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780204669 0 0) 0x7f6e400d0140 con 0x7f6e400ce830 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e280035b0 con 0x7f6e400ce830 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1856838818 0 0) 0x7f6e280035b0 con 0x7f6e400ce830 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 >> v2:172.21.15.172:3300/0 conn(0x7f6e4015bf50 msgr2=0x7f6e400ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 --2- 172.21.15.114:0/1022283802 >> v2:172.21.15.172:3300/0 conn(0x7f6e4015bf50 0x7f6e400ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 >> v1:172.21.15.172:6789/0 conn(0x7f6e400c6c30 legacy=0x7f6e400cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.933 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400d03e0 con 0x7f6e400ce830 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f6e34001670 con 0x7f6e400ce830 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 >> v1:172.21.15.114:6789/0 conn(0x7f6e400ce830 legacy=0x7f6e400cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e28003a00 0x7f6e28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e28004400 0x7f6e280048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 0x7f6e2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.934 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4015c330 con 0x7f6e280050d0 2023-11-24T06:25:31.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2800c030 con 0x7f6e28003a00 2023-11-24T06:25:31.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2800c1f0 con 0x7f6e28004400 2023-11-24T06:25:31.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 0x7f6e2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.919+0000 7f6e3e59c700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e28003a00 0x7f6e28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3dd9b700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e28004400 0x7f6e280048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e28004400 msgr2=0x7f6e280048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:31.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e28004400 0x7f6e280048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e28003a00 msgr2=0x7f6e28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:31.937 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e28003a00 0x7f6e28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.937 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/1022283802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280035b0 con 0x7f6e280050d0 2023-11-24T06:25:31.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/1022283802 reap_dead start 2023-11-24T06:25:31.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 0x7f6e2800ba10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c260 tx=0x7f6e28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94098996ac2c2c65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38001e70 con 0x7f6e280050d0 2023-11-24T06:25:31.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3800ce00 con 0x7f6e280050d0 2023-11-24T06:25:31.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e3d59a700 1 -- 172.21.15.114:0/1022283802 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38003c40 con 0x7f6e280050d0 2023-11-24T06:25:31.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e44fe7700 1 -- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 msgr2=0x7f6e2800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.920+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 0x7f6e2800ba10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c260 tx=0x7f6e28005880 comp rx=0 tx=0).stop 2023-11-24T06:25:31.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.921+0000 7f6e44fe7700 1 -- 172.21.15.114:0/1022283802 shutdown_connections 2023-11-24T06:25:31.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.921+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/1022283802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e280050d0 0x7f6e2800ba10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.921+0000 7f6e44fe7700 1 -- 172.21.15.114:0/1022283802 >> 172.21.15.114:0/1022283802 conn(0x7f6e400b3780 msgr2=0x7f6e400b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:31.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.921+0000 7f6e44fe7700 1 -- 172.21.15.114:0/1022283802 shutdown_connections 2023-11-24T06:25:31.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.921+0000 7f6e44fe7700 1 -- 172.21.15.114:0/1022283802 wait complete. 2023-11-24T06:25:31.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.922+0000 7f6e44fe7700 1 Processor -- start 2023-11-24T06:25:31.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.922+0000 7f6e44fe7700 1 -- start start 2023-11-24T06:25:31.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e280050d0 0x7f6e401f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e401f7be0 0x7f6e4006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4006d050 con 0x7f6e4006c2a0 2023-11-24T06:25:31.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4006ca90 con 0x7f6e401f7be0 2023-11-24T06:25:31.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4006cd70 con 0x7f6e280050d0 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40196/0 (socket says 172.21.15.114:40196) 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/954443279 learned_addr learned my addr 172.21.15.114:0/954443279 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3e59c700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e280050d0 0x7f6e401f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3dd9b700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e401f7be0 0x7f6e4006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e280050d0 msgr2=0x7f6e401f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e280050d0 0x7f6e401f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e401f7be0 msgr2=0x7f6e4006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e401f7be0 0x7f6e4006bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 -- 172.21.15.114:0/954443279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400c9e00 con 0x7f6e4006c2a0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e3ed9d700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6e400cc190 tx=0x7f6e380015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3801a050 con 0x7f6e4006c2a0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38018030 con 0x7f6e4006c2a0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400ca0e0 con 0x7f6e4006c2a0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38005520 con 0x7f6e4006c2a0 2023-11-24T06:25:31.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.923+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e400b4870 con 0x7f6e4006c2a0 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.924+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e400ca2a0 con 0x7f6e4006c2a0 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.925+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6e38005680 con 0x7f6e4006c2a0 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.925+0000 7f6e2effd700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 0x7f6e2006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.925+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e3808a210 con 0x7f6e4006c2a0 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.925+0000 7f6e3e59c700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 0x7f6e2006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:31.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.926+0000 7f6e3e59c700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 0x7f6e2006d540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009be0 tx=0x7f6e30009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:31.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:31.930+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e380552b0 con 0x7f6e4006c2a0 2023-11-24T06:25:32.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.119+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e40068fb0 con 0x7f6e4006c2a0 2023-11-24T06:25:32.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.120+0000 7f6e2effd700 1 -- 172.21.15.114:0/954443279 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e380588d0 con 0x7f6e4006c2a0 2023-11-24T06:25:32.123 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:32.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 msgr2=0x7f6e2006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 0x7f6e2006d540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009be0 tx=0x7f6e30009210 comp rx=0 tx=0).stop 2023-11-24T06:25:32.125 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 msgr2=0x7f6e400b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6e400cc190 tx=0x7f6e380015b0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 shutdown_connections 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6e2006b080 0x7f6e2006d540 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e4006c2a0 0x7f6e400b5b40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e280050d0 0x7f6e401f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 --2- 172.21.15.114:0/954443279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e401f7be0 0x7f6e4006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 >> 172.21.15.114:0/954443279 conn(0x7f6e400b3780 msgr2=0x7f6e400cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 shutdown_connections 2023-11-24T06:25:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.122+0000 7f6e44fe7700 1 -- 172.21.15.114:0/954443279 wait complete. 2023-11-24T06:25:32.138 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:25:32.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.256+0000 7f4e4ca9d700 1 Processor -- start 2023-11-24T06:25:32.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.256+0000 7f4e4ca9d700 1 -- start start 2023-11-24T06:25:32.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4ca9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4e481446a0 0x7f4e480b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4ca9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e480adf00 0x7f4e4806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4ca9d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480af950 con 0x7f4e4806c540 2023-11-24T06:25:32.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4ca9d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4806c9f0 con 0x7f4e481446a0 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4ca9d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e48144a80 con 0x7f4e480adf00 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4e481446a0 0x7f4e480b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e46d9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4e4806c540 0x7f4e4806da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55912/0 (socket says 172.21.15.114:55912) 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e46d9d700 1 -- 172.21.15.114:0/2568011040 learned_addr learned my addr 172.21.15.114:0/2568011040 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e45d9b700 1 --2- 172.21.15.114:0/2568011040 >> v2:172.21.15.172:3300/0 conn(0x7f4e480adf00 0x7f4e4806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 -- 172.21.15.114:0/2568011040 >> v2:172.21.15.172:3300/0 conn(0x7f4e480adf00 msgr2=0x7f4e4806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 --2- 172.21.15.114:0/2568011040 >> v2:172.21.15.172:3300/0 conn(0x7f4e480adf00 0x7f4e4806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 -- 172.21.15.114:0/2568011040 >> v1:172.21.15.114:6789/0 conn(0x7f4e4806c540 legacy=0x7f4e4806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 -- 172.21.15.114:0/2568011040 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480afc30 con 0x7f4e481446a0 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4559a700 1 -- 172.21.15.114:0/2568011040 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759812168 0 0) 0x7f4e480af950 con 0x7f4e4806c540 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4659c700 1 --2- 172.21.15.114:0/2568011040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e481446a0 0x7f4e480b5ae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e4809f030 tx=0x7f4e38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aea8e396e963f09 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.257+0000 7f4e4559a700 1 -- 172.21.15.114:0/2568011040 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011830 con 0x7f4e481446a0 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4559a700 1 -- 172.21.15.114:0/2568011040 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38011990 con 0x7f4e481446a0 2023-11-24T06:25:32.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4559a700 1 -- 172.21.15.114:0/2568011040 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011d20 con 0x7f4e481446a0 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/2568011040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e481446a0 msgr2=0x7f4e480b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/2568011040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e481446a0 0x7f4e480b5ae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4e4809f030 tx=0x7f4e38009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/2568011040 shutdown_connections 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/2568011040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e481446a0 0x7f4e480b5ae0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/2568011040 >> v2:172.21.15.172:3300/0 conn(0x7f4e480adf00 0x7f4e4806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/2568011040 >> 172.21.15.114:0/2568011040 conn(0x7f4e48071b90 msgr2=0x7f4e48071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:32.265 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/2568011040 shutdown_connections 2023-11-24T06:25:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.258+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/2568011040 wait complete. 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.259+0000 7f4e4ca9d700 1 Processor -- start 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.259+0000 7f4e4ca9d700 1 -- start start 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4806c540 0x7f4e48158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e481446a0 0x7f4e48159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4809b290 con 0x7f4e480adf00 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4809b570 con 0x7f4e4806c540 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4ca9d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4809b850 con 0x7f4e481446a0 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e45d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e45d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40208/0 (socket says 172.21.15.114:40208) 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e45d9b700 1 -- 172.21.15.114:0/1553809944 learned_addr learned my addr 172.21.15.114:0/1553809944 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:32.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.260+0000 7f4e4659c700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4806c540 0x7f4e48158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e46d9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e481446a0 0x7f4e48159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 -- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e481446a0 msgr2=0x7f4e48159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e481446a0 0x7f4e48159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 -- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4806c540 msgr2=0x7f4e48158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4806c540 0x7f4e48158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 -- 172.21.15.114:0/1553809944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380094a0 con 0x7f4e480adf00 2023-11-24T06:25:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e45d9b700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e7e0 tx=0x7f4e3c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c013610 con 0x7f4e480adf00 2023-11-24T06:25:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c013770 con 0x7f4e480adf00 2023-11-24T06:25:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4809bb30 con 0x7f4e480adf00 2023-11-24T06:25:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c013b20 con 0x7f4e480adf00 2023-11-24T06:25:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.261+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48166760 con 0x7f4e480adf00 2023-11-24T06:25:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.262+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4809c180 con 0x7f4e480adf00 2023-11-24T06:25:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.262+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4e3c013c80 con 0x7f4e480adf00 2023-11-24T06:25:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.263+0000 7f4e36ffd700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 0x7f4e2806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.263+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e3c0896c0 con 0x7f4e480adf00 2023-11-24T06:25:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.263+0000 7f4e4659c700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 0x7f4e2806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.263+0000 7f4e4659c700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 0x7f4e2806d540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e380090b0 tx=0x7f4e3801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.268+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e3c054340 con 0x7f4e480adf00 2023-11-24T06:25:32.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.473+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e480753f0 con 0x7f4e480adf00 2023-11-24T06:25:32.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.474+0000 7f4e36ffd700 1 -- 172.21.15.114:0/1553809944 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e3c057960 con 0x7f4e480adf00 2023-11-24T06:25:32.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 msgr2=0x7f4e2806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 0x7f4e2806d540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e380090b0 tx=0x7f4e3801c040 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 msgr2=0x7f4e481590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e7e0 tx=0x7f4e3c00eba0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 shutdown_connections 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4e2806b080 0x7f4e2806d540 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e481446a0 0x7f4e48159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e4806c540 0x7f4e48158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 --2- 172.21.15.114:0/1553809944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e480adf00 0x7f4e481590e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 >> 172.21.15.114:0/1553809944 conn(0x7f4e48071b90 msgr2=0x7f4e480b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 shutdown_connections 2023-11-24T06:25:32.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.476+0000 7f4e4ca9d700 1 -- 172.21.15.114:0/1553809944 wait complete. 2023-11-24T06:25:32.481 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:25:32.491 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:25:32.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:32.492 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:25:32.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:25:32.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:32.576 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:25:32.612 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:32.612 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:32.631 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:32.632 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:32.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:32.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.808+0000 7f1ec77d9700 1 Processor -- start 2023-11-24T06:25:32.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.808+0000 7f1ec77d9700 1 -- start start 2023-11-24T06:25:32.814 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec77d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1ec00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec77d9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ec00cfd40 0x7f1ec00b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec77d9700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec00b4290 con 0x7f1ec013e000 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec77d9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec013e570 con 0x7f1ec00cfd40 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec77d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00d05d0 con 0x7f1ec015f180 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec5575700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1ec013e000 0x7f1ec015ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55924/0 (socket says 172.21.15.114:55924) 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec5575700 1 -- 172.21.15.114:0/1776651012 learned_addr learned my addr 172.21.15.114:0/1776651012 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec4d74700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1ec00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec5d76700 1 --2- 172.21.15.114:0/1776651012 >> v2:172.21.15.172:3300/0 conn(0x7f1ec00cfd40 0x7f1ec00b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477418906 0 0) 0x7f1ec00b4290 con 0x7f1ec013e000 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac0035b0 con 0x7f1ec013e000 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102697100 0 0) 0x7f1eac0035b0 con 0x7f1ec013e000 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 msgr2=0x7f1ec00cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1ec00cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 >> v2:172.21.15.172:3300/0 conn(0x7f1ec00cfd40 msgr2=0x7f1ec00b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 --2- 172.21.15.114:0/1776651012 >> v2:172.21.15.172:3300/0 conn(0x7f1ec00cfd40 0x7f1ec00b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00d08b0 con 0x7f1ec013e000 2023-11-24T06:25:32.816 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f1ebc001670 con 0x7f1ec013e000 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1ec5d76700 1 --2- 172.21.15.114:0/1776651012 >> v2:172.21.15.172:3300/0 conn(0x7f1ec00cfd40 0x7f1ec00b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 >> v1:172.21.15.114:6789/0 conn(0x7f1ec013e000 legacy=0x7f1ec015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eac003a00 0x7f1eac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1eac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eac004d90 0x7f1eac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.809+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec013e570 con 0x7f1eac003a00 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00d05d0 con 0x7f1eac004d90 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac00ba10 con 0x7f1ec015f180 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eac003a00 0x7f1eac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec4d74700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1eac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5d76700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eac004d90 0x7f1eac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 -- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 msgr2=0x7f1eac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:32.818 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ec015f180 0x7f1eac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 -- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eac004d90 msgr2=0x7f1eac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eac004d90 0x7f1eac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 -- 172.21.15.114:0/1776651012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac0035b0 con 0x7f1eac003a00 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5d76700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eac004d90 0x7f1eac005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eac003a00 0x7f1eac003e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c320 tx=0x7f1eac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbb1c616d9e7495 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00ce60 con 0x7f1eac003a00 2023-11-24T06:25:32.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc005960 con 0x7f1eac003a00 2023-11-24T06:25:32.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.810+0000 7f1eb7fff700 1 -- 172.21.15.114:0/1776651012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc005cf0 con 0x7f1eac003a00 2023-11-24T06:25:32.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1776651012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eac003a00 msgr2=0x7f1eac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1776651012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eac003a00 0x7f1eac003e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c320 tx=0x7f1eac005880 comp rx=0 tx=0).stop 2023-11-24T06:25:32.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec5d76700 1 -- 172.21.15.114:0/1776651012 reap_dead start 2023-11-24T06:25:32.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1776651012 shutdown_connections 2023-11-24T06:25:32.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1776651012 >> 172.21.15.114:0/1776651012 conn(0x7f1ec0072990 msgr2=0x7f1ec006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:32.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1776651012 shutdown_connections 2023-11-24T06:25:32.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.811+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1776651012 wait complete. 2023-11-24T06:25:32.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.812+0000 7f1ec77d9700 1 Processor -- start 2023-11-24T06:25:32.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.812+0000 7f1ec77d9700 1 -- start start 2023-11-24T06:25:32.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eac003a00 0x7f1ec013d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec0143380 0x7f1ec015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0157f70 con 0x7f1ec0157af0 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0156a50 con 0x7f1ec0143380 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0156d30 con 0x7f1eac003a00 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40230/0 (socket says 172.21.15.114:40230) 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 -- 172.21.15.114:0/1705829216 learned_addr learned my addr 172.21.15.114:0/1705829216 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:32.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eac003a00 0x7f1ec013d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec5d76700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec0143380 0x7f1ec015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 -- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eac003a00 msgr2=0x7f1ec013d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eac003a00 0x7f1ec013d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 -- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec0143380 msgr2=0x7f1ec015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec0143380 0x7f1ec015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:32.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 -- 172.21.15.114:0/1705829216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0157010 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec4d74700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0011aa0 tx=0x7f1eb0011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0011ee0 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000f940 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01572f0 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.813+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000fcf0 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.814+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec0144140 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.815+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0157940 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.815+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f1eb001d400 con 0x7f1ec0157af0 2023-11-24T06:25:32.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.815+0000 7f1eb5ffb700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 0x7f1ea406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:32.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.815+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eb0089f20 con 0x7f1ec0157af0 2023-11-24T06:25:32.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.816+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 0x7f1ea406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:32.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.816+0000 7f1ec5575700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 0x7f1ea406d540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc008fd0 tx=0x7f1ebc00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:32.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:32.820+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1eb0054e10 con 0x7f1ec0157af0 2023-11-24T06:25:33.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.011+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ec0067f60 con 0x7f1ec0157af0 2023-11-24T06:25:33.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.013+0000 7f1eb5ffb700 1 -- 172.21.15.114:0/1705829216 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f1eb0058430 con 0x7f1ec0157af0 2023-11-24T06:25:33.015 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:33.015 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:33.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 msgr2=0x7f1ea406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 0x7f1ea406d540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc008fd0 tx=0x7f1ebc00a000 comp rx=0 tx=0).stop 2023-11-24T06:25:33.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 msgr2=0x7f1ec0142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0011aa0 tx=0x7f1eb0011db0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 shutdown_connections 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1ea406b080 0x7f1ea406d540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ec0143380 0x7f1ec015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1eac003a00 0x7f1ec013d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 --2- 172.21.15.114:0/1705829216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ec0157af0 0x7f1ec0142e40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 >> 172.21.15.114:0/1705829216 conn(0x7f1ec0072990 msgr2=0x7f1ec00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 shutdown_connections 2023-11-24T06:25:33.018 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.015+0000 7f1ec77d9700 1 -- 172.21.15.114:0/1705829216 wait complete. 2023-11-24T06:25:33.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:15.659348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:16.837675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:33.030 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:33.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.151+0000 7f931cd61700 1 Processor -- start 2023-11-24T06:25:33.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.151+0000 7f931cd61700 1 -- start start 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931cd61700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f931806c000 0x7f931806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931cd61700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f931806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931cd61700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180b3fb0 con 0x7f931815e3e0 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931cd61700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180b56d0 con 0x7f931806c000 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931cd61700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931806c920 con 0x7f93180b5ee0 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931659c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f931806c000 0x7f931806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f9315d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f931815e3e0 0x7f93180b4fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55944/0 (socket says 172.21.15.114:55944) 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f9315d9b700 1 -- 172.21.15.114:0/2319954112 learned_addr learned my addr 172.21.15.114:0/2319954112 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f9316d9d700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f931806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931659c700 1 -- 172.21.15.114:0/2319954112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 msgr2=0x7f931806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931659c700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f931806e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.152+0000 7f931659c700 1 -- 172.21.15.114:0/2319954112 >> v1:172.21.15.114:6789/0 conn(0x7f931815e3e0 legacy=0x7f93180b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931659c700 1 -- 172.21.15.114:0/2319954112 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180b4200 con 0x7f931806c000 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931659c700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931806c000 0x7f931806c3e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f930c009840 tx=0x7f930c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c2b6cd77d5c1f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931559a700 1 -- 172.21.15.114:0/2319954112 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c011830 con 0x7f931806c000 2023-11-24T06:25:33.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931559a700 1 -- 172.21.15.114:0/2319954112 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c011990 con 0x7f931806c000 2023-11-24T06:25:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931559a700 1 -- 172.21.15.114:0/2319954112 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c011d20 con 0x7f931806c000 2023-11-24T06:25:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 -- 172.21.15.114:0/2319954112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931806c000 msgr2=0x7f931806c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931806c000 0x7f931806c3e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f930c009840 tx=0x7f930c009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 -- 172.21.15.114:0/2319954112 shutdown_connections 2023-11-24T06:25:33.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f931806e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 --2- 172.21.15.114:0/2319954112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931806c000 0x7f931806c3e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 -- 172.21.15.114:0/2319954112 >> 172.21.15.114:0/2319954112 conn(0x7f9318071ac0 msgr2=0x7f9318071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 -- 172.21.15.114:0/2319954112 shutdown_connections 2023-11-24T06:25:33.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.153+0000 7f931cd61700 1 -- 172.21.15.114:0/2319954112 wait complete. 2023-11-24T06:25:33.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 Processor -- start 2023-11-24T06:25:33.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 -- start start 2023-11-24T06:25:33.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 0x7f93180ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f93180ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180d0070 con 0x7f931815e3e0 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cf460 con 0x7f931806c000 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.155+0000 7f931cd61700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cf740 con 0x7f93180b5ee0 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40244/0 (socket says 172.21.15.114:40244) 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 -- 172.21.15.114:0/1593046905 learned_addr learned my addr 172.21.15.114:0/1593046905 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:33.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9315d9b700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f93180ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f931659c700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 0x7f93180ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 -- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 msgr2=0x7f93180ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f93180ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 -- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 msgr2=0x7f93180ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 0x7f93180ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 -- 172.21.15.114:0/1593046905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0094a0 con 0x7f931815e3e0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f931659c700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 0x7f93180ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9316d9d700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f931000efc0 tx=0x7f931000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000cb80 con 0x7f931815e3e0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000f040 con 0x7f931815e3e0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180cfa20 con 0x7f931815e3e0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310014480 con 0x7f931815e3e0 2023-11-24T06:25:33.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.156+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180cf050 con 0x7f931815e3e0 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.157+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318068f10 con 0x7f931815e3e0 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.158+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f93100145e0 con 0x7f931815e3e0 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.159+0000 7f9306ffd700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 0x7f92f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.159+0000 7f931659c700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 0x7f92f806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.159+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f931008a920 con 0x7f931815e3e0 2023-11-24T06:25:33.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.159+0000 7f931659c700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 0x7f92f806d570 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.163+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9310055690 con 0x7f931815e3e0 2023-11-24T06:25:33.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.354+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9318072220 con 0x7f931815e3e0 2023-11-24T06:25:33.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.354+0000 7f9306ffd700 1 -- 172.21.15.114:0/1593046905 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93100180a0 con 0x7f931815e3e0 2023-11-24T06:25:33.357 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:33.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 msgr2=0x7f92f806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 0x7f92f806d570 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c01c040 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 msgr2=0x7f931814cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f931000efc0 tx=0x7f931000c0c0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 shutdown_connections 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f92f806b0b0 0x7f92f806d570 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f931815e3e0 0x7f931814cfd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931806c000 0x7f93180ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 --2- 172.21.15.114:0/1593046905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93180b5ee0 0x7f93180ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.356+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 >> 172.21.15.114:0/1593046905 conn(0x7f9318071ac0 msgr2=0x7f9318073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.357+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 shutdown_connections 2023-11-24T06:25:33.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.357+0000 7f931cd61700 1 -- 172.21.15.114:0/1593046905 wait complete. 2023-11-24T06:25:33.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:33.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.493+0000 7fa6e97e1700 1 Processor -- start 2023-11-24T06:25:33.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.493+0000 7fa6e97e1700 1 -- start start 2023-11-24T06:25:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6e415e140 0x7fa6e40d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 0x7fa6e40d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 0x7fa6e40b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e415e520 con 0x7fa6e415e140 2023-11-24T06:25:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e40d2930 con 0x7fa6e40b98c0 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e97e1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e40d2c10 con 0x7fa6e40d39d0 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6e415e140 0x7fa6e40d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa6e415e140 0x7fa6e40d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40248/0 (socket says 172.21.15.114:40248) 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/26807877 learned_addr learned my addr 172.21.15.114:0/26807877 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e27fc700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 0x7fa6e40d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e37fe700 1 --2- 172.21.15.114:0/26807877 >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 0x7fa6e40b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/26807877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 msgr2=0x7fa6e40d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 0x7fa6e40d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/26807877 >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 msgr2=0x7fa6e40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/26807877 >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 0x7fa6e40b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/26807877 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40d2ef0 con 0x7fa6e415e140 2023-11-24T06:25:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e37fe700 1 --2- 172.21.15.114:0/26807877 >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 0x7fa6e40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e27fc700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 0x7fa6e40d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.494+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e415e140 0x7fa6e40d3ec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40d4f70 tx=0x7fa6d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52a64f70fef6203 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e1ffb700 1 -- 172.21.15.114:0/26807877 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800b870 con 0x7fa6e415e140 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e1ffb700 1 -- 172.21.15.114:0/26807877 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d800b9d0 con 0x7fa6e415e140 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e1ffb700 1 -- 172.21.15.114:0/26807877 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800bd60 con 0x7fa6e415e140 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 -- 172.21.15.114:0/26807877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e415e140 msgr2=0x7fa6e40d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e415e140 0x7fa6e40d3ec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40d4f70 tx=0x7fa6d8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 -- 172.21.15.114:0/26807877 shutdown_connections 2023-11-24T06:25:33.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/26807877 >> v2:172.21.15.172:3300/0 conn(0x7fa6e40b98c0 0x7fa6e40b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e415e140 0x7fa6e40d3ec0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/26807877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e40d39d0 0x7fa6e40d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 -- 172.21.15.114:0/26807877 >> 172.21.15.114:0/26807877 conn(0x7fa6e406b410 msgr2=0x7fa6e406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 -- 172.21.15.114:0/26807877 shutdown_connections 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.495+0000 7fa6e97e1700 1 -- 172.21.15.114:0/26807877 wait complete. 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.496+0000 7fa6e97e1700 1 Processor -- start 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 -- start start 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e40d39d0 0x7fa6e40b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e415e140 0x7fa6e40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e40b47f0 con 0x7fa6e40b98c0 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e40b3890 con 0x7fa6e40d39d0 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e97e1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e40b3b70 con 0x7fa6e415e140 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40254/0 (socket says 172.21.15.114:40254) 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/3231191434 learned_addr learned my addr 172.21.15.114:0/3231191434 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.497+0000 7fa6e27fc700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e40d39d0 0x7fa6e40b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e37fe700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e415e140 0x7fa6e40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e415e140 msgr2=0x7fa6e40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e415e140 0x7fa6e40c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e40d39d0 msgr2=0x7fa6e40b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e40d39d0 0x7fa6e40b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 -- 172.21.15.114:0/3231191434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8009550 con 0x7fa6e40b98c0 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e2ffd700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8000c00 tx=0x7fa6d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8004520 con 0x7fa6e40b98c0 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d801e070 con 0x7fa6e40b98c0 2023-11-24T06:25:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e40b3e50 con 0x7fa6e40b98c0 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80038c0 con 0x7fa6e40b98c0 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.498+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e40b4410 con 0x7fa6e40b98c0 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.499+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e40d22f0 con 0x7fa6e40b98c0 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.499+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa6d8003a20 con 0x7fa6e40b98c0 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.500+0000 7fa6cb7fe700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 0x7fa6d006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.500+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6d8020070 con 0x7fa6e40b98c0 2023-11-24T06:25:33.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.500+0000 7fa6e27fc700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 0x7fa6d006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.500+0000 7fa6e27fc700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 0x7fa6d006d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40c9660 tx=0x7fa6d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.504+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6d8055d10 con 0x7fa6e40b98c0 2023-11-24T06:25:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.693+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6e40d1bb0 con 0x7fa6e40b98c0 2023-11-24T06:25:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.693+0000 7fa6cb7fe700 1 -- 172.21.15.114:0/3231191434 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6d80150a0 con 0x7fa6e40b98c0 2023-11-24T06:25:33.696 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 msgr2=0x7fa6d006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 0x7fa6d006d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40c9660 tx=0x7fa6d4008040 comp rx=0 tx=0).stop 2023-11-24T06:25:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 msgr2=0x7fa6e40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8000c00 tx=0x7fa6d8004750 comp rx=0 tx=0).stop 2023-11-24T06:25:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 shutdown_connections 2023-11-24T06:25:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa6d006b080 0x7fa6d006d540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa6e415e140 0x7fa6e40c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6e40b98c0 0x7fa6e40b4c10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 --2- 172.21.15.114:0/3231191434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6e40d39d0 0x7fa6e40b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 >> 172.21.15.114:0/3231191434 conn(0x7fa6e406b410 msgr2=0x7fa6e415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 shutdown_connections 2023-11-24T06:25:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.695+0000 7fa6e97e1700 1 -- 172.21.15.114:0/3231191434 wait complete. 2023-11-24T06:25:33.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:33.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.831+0000 7f499b2fe700 1 Processor -- start 2023-11-24T06:25:33.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.831+0000 7f499b2fe700 1 -- start start 2023-11-24T06:25:33.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499b2fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f499415e160 0x7f49940b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499b2fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 0x7f499406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499b2fe700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49940b3c60 con 0x7f49940afc90 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499b2fe700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49940b4500 con 0x7f499415e160 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499b2fe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940b58c0 con 0x7f49940746a0 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f4998899700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f49940afc90 0x7f4994073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55970/0 (socket says 172.21.15.114:55970) 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f4998899700 1 -- 172.21.15.114:0/4250701835 learned_addr learned my addr 172.21.15.114:0/4250701835 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499989b700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 0x7f499406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499909a700 1 --2- 172.21.15.114:0/4250701835 >> v2:172.21.15.172:3300/0 conn(0x7f499415e160 0x7f49940b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129535483 0 0) 0x7f49940b3c60 con 0x7f49940afc90 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49800035b0 con 0x7f49940afc90 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493975447 0 0) 0x7f49800035b0 con 0x7f49940afc90 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499909a700 1 -- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 msgr2=0x7f499406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.832+0000 7f499909a700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 0x7f499406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499909a700 1 -- 172.21.15.114:0/4250701835 >> v1:172.21.15.114:6789/0 conn(0x7f49940afc90 legacy=0x7f4994073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499909a700 1 -- 172.21.15.114:0/4250701835 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49940b0140 con 0x7f499415e160 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499989b700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 0x7f499406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499909a700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f499415e160 0x7f49940b5da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4990009840 tx=0x7f4990009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2f856c610510d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984001670 con 0x7f499415e160 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4990004030 con 0x7f499415e160 2023-11-24T06:25:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f498bfff700 1 -- 172.21.15.114:0/4250701835 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900043c0 con 0x7f499415e160 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 -- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f499415e160 msgr2=0x7f49940b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f499415e160 0x7f49940b5da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4990009840 tx=0x7f4990009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 -- 172.21.15.114:0/4250701835 shutdown_connections 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49940746a0 0x7f499406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 --2- 172.21.15.114:0/4250701835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f499415e160 0x7f49940b5da0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.833+0000 7f499b2fe700 1 -- 172.21.15.114:0/4250701835 >> 172.21.15.114:0/4250701835 conn(0x7f49940d0bf0 msgr2=0x7f49940d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.834+0000 7f499b2fe700 1 -- 172.21.15.114:0/4250701835 shutdown_connections 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.834+0000 7f499b2fe700 1 -- 172.21.15.114:0/4250701835 wait complete. 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 Processor -- start 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 -- start start 2023-11-24T06:25:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 0x7f49941f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 0x7f49941602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c5c70 con 0x7f49940746a0 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c5f50 con 0x7f49940afc90 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.835+0000 7f499b2fe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c6230 con 0x7f499415e160 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40268/0 (socket says 172.21.15.114:40268) 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 -- 172.21.15.114:0/2841932621 learned_addr learned my addr 172.21.15.114:0/2841932621 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f4998899700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 0x7f49941f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499989b700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 0x7f49941602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 -- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 msgr2=0x7f49941602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 0x7f49941602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 -- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 msgr2=0x7f49941f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 0x7f49941f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 -- 172.21.15.114:0/2841932621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c009a70 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f4998899700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 0x7f49941f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499989b700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 0x7f49941602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499909a700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f499000a010 tx=0x7f4990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990003c40 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499001f070 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900094a0 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011630 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.836+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49940c67f0 con 0x7f49940746a0 2023-11-24T06:25:33.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.837+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49940b54a0 con 0x7f49940746a0 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.838+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4990021030 con 0x7f49940746a0 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.838+0000 7f4989ffb700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 0x7f497806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.838+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f499008aca0 con 0x7f49940746a0 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.838+0000 7f4998899700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 0x7f497806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.838+0000 7f4998899700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 0x7f497806d540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4994074d70 tx=0x7f4984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:33.842+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4990055b60 con 0x7f49940746a0 2023-11-24T06:25:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.033+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4994068fb0 con 0x7f49940746a0 2023-11-24T06:25:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.033+0000 7f4989ffb700 1 -- 172.21.15.114:0/2841932621 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4990059180 con 0x7f49940746a0 2023-11-24T06:25:34.043 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 msgr2=0x7f497806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 0x7f497806d540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4994074d70 tx=0x7f4984009040 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 msgr2=0x7f4994158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f499000a010 tx=0x7f4990004750 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 shutdown_connections 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f497806b080 0x7f497806d540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f499415e160 0x7f49941602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49940746a0 0x7f4994158860 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 --2- 172.21.15.114:0/2841932621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49940afc90 0x7f49941f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 >> 172.21.15.114:0/2841932621 conn(0x7f49940d0bf0 msgr2=0x7f4994162530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 shutdown_connections 2023-11-24T06:25:34.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.035+0000 7f499b2fe700 1 -- 172.21.15.114:0/2841932621 wait complete. 2023-11-24T06:25:34.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:25:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.171+0000 7f3c9b214700 1 Processor -- start 2023-11-24T06:25:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.171+0000 7f3c9b214700 1 -- start start 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c9b214700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158a50 0x7f3c940c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c9b214700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3c94162a30 0x7f3c940c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c9b214700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c940cfff0 con 0x7f3c94158670 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c9b214700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c940629b0 con 0x7f3c94162a30 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c9b214700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940cfd10 con 0x7f3c94158a50 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3c94162a30 0x7f3c940c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3c94162a30 0x7f3c940c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40278/0 (socket says 172.21.15.114:40278) 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 -- 172.21.15.114:0/4276407813 learned_addr learned my addr 172.21.15.114:0/4276407813 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:34.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 -- 172.21.15.114:0/4276407813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158a50 msgr2=0x7f3c940c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 --2- 172.21.15.114:0/4276407813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158a50 0x7f3c940c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 -- 172.21.15.114:0/4276407813 >> v1:172.21.15.172:6789/0 conn(0x7f3c94158670 legacy=0x7f3c940622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 -- 172.21.15.114:0/4276407813 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94161f50 con 0x7f3c94162a30 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.172+0000 7f3c997b1700 1 --2- 172.21.15.114:0/4276407813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940c9df0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009e50 tx=0x7f3c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c89db89cb7b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c937fe700 1 -- 172.21.15.114:0/4276407813 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00e070 con 0x7f3c94162a30 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c937fe700 1 -- 172.21.15.114:0/4276407813 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c003a20 con 0x7f3c94162a30 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c937fe700 1 -- 172.21.15.114:0/4276407813 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c003dd0 con 0x7f3c94162a30 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 -- 172.21.15.114:0/4276407813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 msgr2=0x7f3c940c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 --2- 172.21.15.114:0/4276407813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940c9df0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009e50 tx=0x7f3c8c00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 -- 172.21.15.114:0/4276407813 shutdown_connections 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 --2- 172.21.15.114:0/4276407813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940c9df0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 --2- 172.21.15.114:0/4276407813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158a50 0x7f3c940c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 -- 172.21.15.114:0/4276407813 >> 172.21.15.114:0/4276407813 conn(0x7f3c940b57b0 msgr2=0x7f3c940b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:34.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 -- 172.21.15.114:0/4276407813 shutdown_connections 2023-11-24T06:25:34.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.173+0000 7f3c9b214700 1 -- 172.21.15.114:0/4276407813 wait complete. 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.174+0000 7f3c9b214700 1 Processor -- start 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.174+0000 7f3c9b214700 1 -- start start 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158670 0x7f3c94158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c94158a50 0x7f3c94154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9415c020 con 0x7f3c94162a30 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9415bd40 con 0x7f3c94158a50 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c9b214700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94155da0 con 0x7f3c94158670 2023-11-24T06:25:34.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c997b1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c997b1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40284/0 (socket says 172.21.15.114:40284) 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c997b1700 1 -- 172.21.15.114:0/193241638 learned_addr learned my addr 172.21.15.114:0/193241638 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c98fb0700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158670 0x7f3c94158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.175+0000 7f3c93fff700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c94158a50 0x7f3c94154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 -- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158670 msgr2=0x7f3c94158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158670 0x7f3c94158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 -- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c94158a50 msgr2=0x7f3c94154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c94158a50 0x7f3c94154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 -- 172.21.15.114:0/193241638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c009b30 con 0x7f3c94162a30 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c997b1700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c940722a0 tx=0x7f3c8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0094d0 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c004240 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c94156080 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0045f0 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.176+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c94159370 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.177+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c94155bd0 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.177+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3c8c019070 con 0x7f3c94162a30 2023-11-24T06:25:34.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.178+0000 7f3c917fa700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 0x7f3c7806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.178+0000 7f3c98fb0700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 0x7f3c7806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.178+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c8c08a870 con 0x7f3c94162a30 2023-11-24T06:25:34.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.178+0000 7f3c98fb0700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 0x7f3c7806d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c88007fd0 tx=0x7f3c88007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.182+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c8c055620 con 0x7f3c94162a30 2023-11-24T06:25:34.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.391+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c94156540 con 0x7f3c94162a30 2023-11-24T06:25:34.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.392+0000 7f3c917fa700 1 -- 172.21.15.114:0/193241638 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c8c020020 con 0x7f3c94162a30 2023-11-24T06:25:34.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 msgr2=0x7f3c7806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 0x7f3c7806d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c88007fd0 tx=0x7f3c88007ec0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 msgr2=0x7f3c940b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c940722a0 tx=0x7f3c8c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 shutdown_connections 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3c7806b080 0x7f3c7806d540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3c94162a30 0x7f3c940b3a00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c94158670 0x7f3c94158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 --2- 172.21.15.114:0/193241638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c94158a50 0x7f3c94154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 >> 172.21.15.114:0/193241638 conn(0x7f3c940b57b0 msgr2=0x7f3c940ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 shutdown_connections 2023-11-24T06:25:34.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.394+0000 7f3c9b214700 1 -- 172.21.15.114:0/193241638 wait complete. 2023-11-24T06:25:34.400 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:25:34.409 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:25:34.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:34.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:25:34.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:25:34.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:25:34.456 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:25:34.475 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:34.475 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:34.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:25:34.493 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:34.493 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:34.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:25:34.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:34.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.653+0000 7f798a201700 1 Processor -- start 2023-11-24T06:25:34.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.653+0000 7f798a201700 1 -- start start 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f798a201700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79840af4d0 0x7f79840b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f798a201700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840ae430 con 0x7f798415e080 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f798a201700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840ae710 con 0x7f798406c710 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f798a201700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79840af0b0 con 0x7f79840af4d0 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79840af4d0 0x7f79840b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f79837fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f798415e080 0x7f798406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55998/0 (socket says 172.21.15.114:55998) 2023-11-24T06:25:34.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f79840af4d0 0x7f79840b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40294/0 (socket says 172.21.15.114:40294) 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f79837fe700 1 -- 172.21.15.114:0/3847482408 learned_addr learned my addr 172.21.15.114:0/3847482408 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 -- 172.21.15.114:0/3847482408 >> v1:172.21.15.172:6789/0 conn(0x7f798406c710 legacy=0x7f798406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 -- 172.21.15.114:0/3847482408 >> v1:172.21.15.114:6789/0 conn(0x7f798415e080 legacy=0x7f798406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 -- 172.21.15.114:0/3847482408 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840ae9f0 con 0x7f79840af4d0 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f79827fc700 1 -- 172.21.15.114:0/3847482408 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172790338 0 0) 0x7f79840ae430 con 0x7f798415e080 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.654+0000 7f7983fff700 1 --2- 172.21.15.114:0/3847482408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840b5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f797c009d90 tx=0x7f797c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43610bc02311f136 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f79827fc700 1 -- 172.21.15.114:0/3847482408 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00e070 con 0x7f79840af4d0 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f79827fc700 1 -- 172.21.15.114:0/3847482408 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00bb90 con 0x7f79840af4d0 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f79827fc700 1 -- 172.21.15.114:0/3847482408 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c003a20 con 0x7f79840af4d0 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 -- 172.21.15.114:0/3847482408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 msgr2=0x7f79840b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 --2- 172.21.15.114:0/3847482408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840b5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f797c009d90 tx=0x7f797c00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 -- 172.21.15.114:0/3847482408 shutdown_connections 2023-11-24T06:25:34.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 --2- 172.21.15.114:0/3847482408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840b5050 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 -- 172.21.15.114:0/3847482408 >> 172.21.15.114:0/3847482408 conn(0x7f7984071b90 msgr2=0x7f7984071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:34.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 -- 172.21.15.114:0/3847482408 shutdown_connections 2023-11-24T06:25:34.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.655+0000 7f798a201700 1 -- 172.21.15.114:0/3847482408 wait complete. 2023-11-24T06:25:34.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.657+0000 7f798a201700 1 Processor -- start 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.657+0000 7f798a201700 1 -- start start 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.657+0000 7f798a201700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f798406c710 0x7f79840b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.657+0000 7f798a201700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f798415e080 0x7f79841520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c9950 con 0x7f79840af4d0 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c8dd0 con 0x7f798415e080 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840c90b0 con 0x7f798406c710 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40300/0 (socket says 172.21.15.114:40300) 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 -- 172.21.15.114:0/3251593035 learned_addr learned my addr 172.21.15.114:0/3251593035 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7983fff700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f798415e080 0x7f79841520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f79837fe700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f798406c710 0x7f79840b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 -- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f798406c710 msgr2=0x7f79840b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f798406c710 0x7f79840b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 -- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f798415e080 msgr2=0x7f79841520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f798415e080 0x7f79841520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 -- 172.21.15.114:0/3251593035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c009a70 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f7982ffd700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f797400c4d0 tx=0x7f797400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974011790 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740118f0 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840c9390 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974011ca0 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.658+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798415b9f0 con 0x7f79840af4d0 2023-11-24T06:25:34.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.659+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840753f0 con 0x7f79840af4d0 2023-11-24T06:25:34.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.660+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f797401c460 con 0x7f79840af4d0 2023-11-24T06:25:34.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.660+0000 7f796bfff700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 0x7f797006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:34.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.660+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79740895d0 con 0x7f79840af4d0 2023-11-24T06:25:34.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.660+0000 7f79837fe700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 0x7f797006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:34.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.660+0000 7f79837fe700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 0x7f797006d540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:34.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.664+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7974054400 con 0x7f79840af4d0 2023-11-24T06:25:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.897+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7984068670 con 0x7f79840af4d0 2023-11-24T06:25:34.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.898+0000 7f796bfff700 1 -- 172.21.15.114:0/3251593035 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f7974057a20 con 0x7f79840af4d0 2023-11-24T06:25:34.900 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:34.900 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 msgr2=0x7f797006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 0x7f797006d540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978007e70 comp rx=0 tx=0).stop 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 msgr2=0x7f79840cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f797400c4d0 tx=0x7f797400c7e0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 shutdown_connections 2023-11-24T06:25:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f797006b080 0x7f797006d540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f798415e080 0x7f79841520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f798406c710 0x7f79840b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 --2- 172.21.15.114:0/3251593035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79840af4d0 0x7f79840cfde0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 >> 172.21.15.114:0/3251593035 conn(0x7f7984071b90 msgr2=0x7f798406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 shutdown_connections 2023-11-24T06:25:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:34.900+0000 7f798a201700 1 -- 172.21.15.114:0/3251593035 wait complete. 2023-11-24T06:25:34.905 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2023-11-24T06:25:34.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:35.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.035+0000 7fd0f9376700 1 Processor -- start 2023-11-24T06:25:35.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.035+0000 7fd0f9376700 1 -- start start 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f9376700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd0f40d2240 0x7fd0f40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f9376700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f40b1d40 con 0x7fd0f415ddd0 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f9376700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f40b28c0 con 0x7fd0f415e1b0 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f9376700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f40b1f90 con 0x7fd0f40d2240 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f27fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd0f40d2240 0x7fd0f40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f2ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd0f415ddd0 0x7fd0f40d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56012/0 (socket says 172.21.15.114:56012) 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f2ffd700 1 -- 172.21.15.114:0/3679616866 learned_addr learned my addr 172.21.15.114:0/3679616866 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.036+0000 7fd0f27fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd0f40d2240 0x7fd0f40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40304/0 (socket says 172.21.15.114:40304) 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3679616866 >> v1:172.21.15.172:6789/0 conn(0x7fd0f415e1b0 legacy=0x7fd0f406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3679616866 >> v1:172.21.15.114:6789/0 conn(0x7fd0f415ddd0 legacy=0x7fd0f40d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3679616866 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40b2270 con 0x7fd0f40d2240 2023-11-24T06:25:35.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f27fc700 1 --2- 172.21.15.114:0/3679616866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f40d2240 0x7fd0f40b1680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009870 tx=0x7fd0e4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07334bb554ba0d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f1ffb700 1 -- 172.21.15.114:0/3679616866 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4011830 con 0x7fd0f40d2240 2023-11-24T06:25:35.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f1ffb700 1 -- 172.21.15.114:0/3679616866 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e4011990 con 0x7fd0f40d2240 2023-11-24T06:25:35.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f1ffb700 1 -- 172.21.15.114:0/3679616866 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4011d20 con 0x7fd0f40d2240 2023-11-24T06:25:35.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f9376700 1 -- 172.21.15.114:0/3679616866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f40d2240 msgr2=0x7fd0f40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.037+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3679616866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f40d2240 0x7fd0f40b1680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009870 tx=0x7fd0e4009b80 comp rx=0 tx=0).stop 2023-11-24T06:25:35.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.038+0000 7fd0f9376700 1 -- 172.21.15.114:0/3679616866 shutdown_connections 2023-11-24T06:25:35.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.038+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3679616866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f40d2240 0x7fd0f40b1680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.038+0000 7fd0f9376700 1 -- 172.21.15.114:0/3679616866 >> 172.21.15.114:0/3679616866 conn(0x7fd0f40731a0 msgr2=0x7fd0f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.038+0000 7fd0f9376700 1 -- 172.21.15.114:0/3679616866 shutdown_connections 2023-11-24T06:25:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.038+0000 7fd0f9376700 1 -- 172.21.15.114:0/3679616866 wait complete. 2023-11-24T06:25:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.039+0000 7fd0f9376700 1 Processor -- start 2023-11-24T06:25:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.039+0000 7fd0f9376700 1 -- start start 2023-11-24T06:25:35.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0f40d2240 0x7fd0f415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0f415e1b0 0x7fd0f41e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f41e8b90 con 0x7fd0f415ddd0 2023-11-24T06:25:35.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f409cad0 con 0x7fd0f415e1b0 2023-11-24T06:25:35.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f41e8e10 con 0x7fd0f40d2240 2023-11-24T06:25:35.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40312/0 (socket says 172.21.15.114:40312) 2023-11-24T06:25:35.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3314352885 learned_addr learned my addr 172.21.15.114:0/3314352885 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:35.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f2ffd700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0f40d2240 0x7fd0f415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f37fe700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0f415e1b0 0x7fd0f41e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0f40d2240 msgr2=0x7fd0f415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0f40d2240 0x7fd0f415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0f415e1b0 msgr2=0x7fd0f41e98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0f415e1b0 0x7fd0f41e98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 -- 172.21.15.114:0/3314352885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40094a0 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f27fc700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40090b0 tx=0x7fd0e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40036a0 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e401f070 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f40b2ed0 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.040+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4003a60 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.041+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f41e93d0 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.042+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f41e9560 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.042+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd0e4003bc0 con 0x7fd0f415ddd0 2023-11-24T06:25:35.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.042+0000 7fd0db7fe700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 0x7fd0e006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.042+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0e4021070 con 0x7fd0f415ddd0 2023-11-24T06:25:35.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.042+0000 7fd0f2ffd700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 0x7fd0e006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.043+0000 7fd0f2ffd700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 0x7fd0e006d570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005910 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.054 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.047+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0e4056050 con 0x7fd0f415ddd0 2023-11-24T06:25:35.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.237+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f4067f60 con 0x7fd0f415ddd0 2023-11-24T06:25:35.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.238+0000 7fd0db7fe700 1 -- 172.21.15.114:0/3314352885 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fd0e4059670 con 0x7fd0f415ddd0 2023-11-24T06:25:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:35.241 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:35.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 msgr2=0x7fd0e006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 0x7fd0e006d570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005910 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 msgr2=0x7fd0f409d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40090b0 tx=0x7fd0e4004750 comp rx=0 tx=0).stop 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 shutdown_connections 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd0e006b0b0 0x7fd0e006d570 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0f415e1b0 0x7fd0f41e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0f40d2240 0x7fd0f415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 --2- 172.21.15.114:0/3314352885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f415ddd0 0x7fd0f409d580 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 >> 172.21.15.114:0/3314352885 conn(0x7fd0f40731a0 msgr2=0x7fd0f415f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 shutdown_connections 2023-11-24T06:25:35.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.241+0000 7fd0f9376700 1 -- 172.21.15.114:0/3314352885 wait complete. 2023-11-24T06:25:35.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:15.659348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:16.837675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:35.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:35.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.377+0000 7f21d8a3c700 1 Processor -- start 2023-11-24T06:25:35.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.377+0000 7f21d8a3c700 1 -- start start 2023-11-24T06:25:35.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 0x7f21d415a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21d40ce150 0x7f21d40622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 0x7f21d406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d406ef90 con 0x7f21d40ce150 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40b2f20 con 0x7f21d40cc540 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d8a3c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40ce960 con 0x7f21d406e5e0 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d1d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21d40ce150 0x7f21d40622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.378+0000 7f21d1d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21d40ce150 0x7f21d40622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40318/0 (socket says 172.21.15.114:40318) 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 -- 172.21.15.114:0/2401972580 learned_addr learned my addr 172.21.15.114:0/2401972580 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d259c700 1 --2- 172.21.15.114:0/2401972580 >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 0x7f21d415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 -- 172.21.15.114:0/2401972580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 msgr2=0x7f21d406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d2d9d700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 0x7f21d406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 0x7f21d406ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 -- 172.21.15.114:0/2401972580 >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 msgr2=0x7f21d415a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/2401972580 >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 0x7f21d415a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 -- 172.21.15.114:0/2401972580 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40cec40 con 0x7f21d40ce150 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d259c700 1 --2- 172.21.15.114:0/2401972580 >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 0x7f21d415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d2d9d700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 0x7f21d406ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:35.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d40ce150 0x7f21d40622a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f21d4072800 tx=0x7f21c400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c18f45aa4705ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d159a700 1 -- 172.21.15.114:0/2401972580 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400dd30 con 0x7f21d40ce150 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d159a700 1 -- 172.21.15.114:0/2401972580 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400de90 con 0x7f21d40ce150 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.379+0000 7f21d159a700 1 -- 172.21.15.114:0/2401972580 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4010690 con 0x7f21d40ce150 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 -- 172.21.15.114:0/2401972580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d40ce150 msgr2=0x7f21d40622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d40ce150 0x7f21d40622a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f21d4072800 tx=0x7f21c400be70 comp rx=0 tx=0).stop 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 -- 172.21.15.114:0/2401972580 shutdown_connections 2023-11-24T06:25:35.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d406e5e0 0x7f21d406ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/2401972580 >> v2:172.21.15.172:3300/0 conn(0x7f21d40cc540 0x7f21d415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/2401972580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d40ce150 0x7f21d40622a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 -- 172.21.15.114:0/2401972580 >> 172.21.15.114:0/2401972580 conn(0x7f21d40b2050 msgr2=0x7f21d40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 -- 172.21.15.114:0/2401972580 shutdown_connections 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.380+0000 7f21d8a3c700 1 -- 172.21.15.114:0/2401972580 wait complete. 2023-11-24T06:25:35.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.381+0000 7f21d8a3c700 1 Processor -- start 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.381+0000 7f21d8a3c700 1 -- start start 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 0x7f21d40ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 0x7f21d40ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d415b7a0 con 0x7f21d406e5e0 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40abf10 con 0x7f21d40cc540 2023-11-24T06:25:35.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d8a3c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d415b9b0 con 0x7f21d40ce150 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40322/0 (socket says 172.21.15.114:40322) 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 -- 172.21.15.114:0/1987062261 learned_addr learned my addr 172.21.15.114:0/1987062261 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 0x7f21d40ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d2d9d700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 0x7f21d40ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 -- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 msgr2=0x7f21d40ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 0x7f21d40ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 -- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 msgr2=0x7f21d40ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 0x7f21d40ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 -- 172.21.15.114:0/1987062261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c400b7f0 con 0x7f21d406e5e0 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 0x7f21d40ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d2d9d700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 0x7f21d40ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.382+0000 7f21d259c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c780 tx=0x7f21c800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8011800 con 0x7f21d406e5e0 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8011960 con 0x7f21d406e5e0 2023-11-24T06:25:35.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d415bc90 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8011cd0 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d415c250 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.383+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4068d90 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.384+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f21c8021350 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.384+0000 7f21c2ffd700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 0x7f21b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.384+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21c8089990 con 0x7f21d406e5e0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.384+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 0x7f21b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.385+0000 7f21d1d9b700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 0x7f21b406d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21c400beb0 tx=0x7f21c4012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.389+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21c80546c0 con 0x7f21d406e5e0 2023-11-24T06:25:35.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.581+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21d4068fb0 con 0x7f21d406e5e0 2023-11-24T06:25:35.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.581+0000 7f21c2ffd700 1 -- 172.21.15.114:0/1987062261 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21c8057ce0 con 0x7f21d406e5e0 2023-11-24T06:25:35.584 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 msgr2=0x7f21b406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 0x7f21b406d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21c400beb0 tx=0x7f21c4012000 comp rx=0 tx=0).stop 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 msgr2=0x7f21d41e5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c780 tx=0x7f21c800cb40 comp rx=0 tx=0).stop 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 shutdown_connections 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21b406b080 0x7f21b406d540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f21d40ce150 0x7f21d40ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21d406e5e0 0x7f21d41e5060 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 --2- 172.21.15.114:0/1987062261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21d40cc540 0x7f21d40ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 >> 172.21.15.114:0/1987062261 conn(0x7f21d40b2050 msgr2=0x7f21d40b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 shutdown_connections 2023-11-24T06:25:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.583+0000 7f21d8a3c700 1 -- 172.21.15.114:0/1987062261 wait complete. 2023-11-24T06:25:35.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:25:35.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.718+0000 7fcd19048700 1 Processor -- start 2023-11-24T06:25:35.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.718+0000 7fcd19048700 1 -- start start 2023-11-24T06:25:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd19048700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcd140b4310 0x7fcd1406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd19048700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406ba50 con 0x7fcd140ce430 2023-11-24T06:25:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd19048700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406b120 con 0x7fcd1415e080 2023-11-24T06:25:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd19048700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd140d1200 con 0x7fcd140b4310 2023-11-24T06:25:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd1359e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcd140ce430 0x7fcd140cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56042/0 (socket says 172.21.15.114:56042) 2023-11-24T06:25:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.719+0000 7fcd1359e700 1 -- 172.21.15.114:0/66765254 learned_addr learned my addr 172.21.15.114:0/66765254 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd1259c700 1 --2- 172.21.15.114:0/66765254 >> v2:172.21.15.172:3300/0 conn(0x7fcd140b4310 0x7fcd1406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807575480 0 0) 0x7fcd1406ba50 con 0x7fcd140ce430 2023-11-24T06:25:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccfc0035b0 con 0x7fcd140ce430 2023-11-24T06:25:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899606684 0 0) 0x7fcd1406b120 con 0x7fcd1415e080 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd1406ba50 con 0x7fcd1415e080 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2507200721 0 0) 0x7fccfc0035b0 con 0x7fcd140ce430 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 >> v2:172.21.15.172:3300/0 conn(0x7fcd140b4310 msgr2=0x7fcd1406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 --2- 172.21.15.114:0/66765254 >> v2:172.21.15.172:3300/0 conn(0x7fcd140b4310 0x7fcd1406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 >> v1:172.21.15.172:6789/0 conn(0x7fcd1415e080 legacy=0x7fcd140d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1406b400 con 0x7fcd140ce430 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fcd0c0019f0 con 0x7fcd140ce430 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 >> v1:172.21.15.114:6789/0 conn(0x7fcd140ce430 legacy=0x7fcd140cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 0x7fccfc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140d1200 con 0x7fccfc003b90 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc00c140 con 0x7fccfc005210 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc00c300 con 0x7fccfc004850 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 0x7fccfc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd1259c700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd1359e700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 -- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccfc004850 msgr2=0x7fccfc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 -- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc005210 msgr2=0x7fccfc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 -- 172.21.15.114:0/66765254 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc0035b0 con 0x7fccfc003b90 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd1359e700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd1359e700 1 -- 172.21.15.114:0/66765254 reap_dead start 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.720+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 0x7fccfc003f90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001940 tx=0x7fccfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1c5038113e3580 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08001430 con 0x7fccfc003b90 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08001d70 con 0x7fccfc003b90 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd11d9b700 1 -- 172.21.15.114:0/66765254 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08005530 con 0x7fccfc003b90 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 -- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 msgr2=0x7fccfc003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 0x7fccfc003f90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001940 tx=0x7fccfc005880 comp rx=0 tx=0).stop 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 -- 172.21.15.114:0/66765254 shutdown_connections 2023-11-24T06:25:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 --2- 172.21.15.114:0/66765254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccfc003b90 0x7fccfc003f90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 -- 172.21.15.114:0/66765254 >> 172.21.15.114:0/66765254 conn(0x7fcd140b3780 msgr2=0x7fcd140b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 -- 172.21.15.114:0/66765254 shutdown_connections 2023-11-24T06:25:35.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.721+0000 7fcd19048700 1 -- 172.21.15.114:0/66765254 wait complete. 2023-11-24T06:25:35.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.723+0000 7fcd19048700 1 Processor -- start 2023-11-24T06:25:35.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.723+0000 7fcd19048700 1 -- start start 2023-11-24T06:25:35.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 0x7fcd1414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd1415f4b0 0x7fcd140b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1415fee0 con 0x7fcd1415f930 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd141601c0 con 0x7fccfc003b90 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd141604a0 con 0x7fcd1415f4b0 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40346/0 (socket says 172.21.15.114:40346) 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 -- 172.21.15.114:0/1560085242 learned_addr learned my addr 172.21.15.114:0/1560085242 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1259c700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd1415f4b0 0x7fcd140b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 0x7fcd1414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 -- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd1415f4b0 msgr2=0x7fcd140b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd1415f4b0 0x7fcd140b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 -- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 msgr2=0x7fcd1414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 0x7fcd1414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 -- 172.21.15.114:0/1560085242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14160780 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 0x7fcd1414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd1359e700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000f80 tx=0x7fcd0c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00eb90 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c00ed10 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd14160a60 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c013660 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.724+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1414f620 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.725+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd14068fb0 con 0x7fcd1415f930 2023-11-24T06:25:35.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.726+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcd0c0137c0 con 0x7fcd1415f930 2023-11-24T06:25:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.726+0000 7fccfb7fe700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 0x7fcd0006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.726+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd0c089e90 con 0x7fcd1415f930 2023-11-24T06:25:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.726+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 0x7fcd0006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.727+0000 7fcd12d9d700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 0x7fcd0006d540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005040 tx=0x7fcd0800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.731+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd0c054c30 con 0x7fcd1415f930 2023-11-24T06:25:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.923+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd140cbf40 con 0x7fcd1415f930 2023-11-24T06:25:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.923+0000 7fccfb7fe700 1 -- 172.21.15.114:0/1560085242 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd0c058250 con 0x7fcd1415f930 2023-11-24T06:25:35.926 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:25:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.925+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 msgr2=0x7fcd0006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.925+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 0x7fcd0006d540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005040 tx=0x7fcd0800a000 comp rx=0 tx=0).stop 2023-11-24T06:25:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.925+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 msgr2=0x7fcd140b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.925+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c000f80 tx=0x7fcd0c00e440 comp rx=0 tx=0).stop 2023-11-24T06:25:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 shutdown_connections 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcd0006b080 0x7fcd0006d540 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd1415f930 0x7fcd140b4db0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccfc003b90 0x7fcd1414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 --2- 172.21.15.114:0/1560085242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd1415f4b0 0x7fcd140b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 >> 172.21.15.114:0/1560085242 conn(0x7fcd140b3780 msgr2=0x7fcd14061aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 shutdown_connections 2023-11-24T06:25:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:35.926+0000 7fcd19048700 1 -- 172.21.15.114:0/1560085242 wait complete. 2023-11-24T06:25:35.940 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:25:36.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 Processor -- start 2023-11-24T06:25:36.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 -- start start 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f00380ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380d0140 con 0x7f00380ce830 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380cf3c0 con 0x7f00380c6c30 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.061+0000 7f003d159700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003815c330 con 0x7f003815bf50 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f003659c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f00380ce830 0x7f00380cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56058/0 (socket says 172.21.15.114:56058) 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f003659c700 1 -- 172.21.15.114:0/1730454197 learned_addr learned my addr 172.21.15.114:0/1730454197 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:36.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140071595 0 0) 0x7f00380d0140 con 0x7f00380ce830 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0036d9d700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f00380ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00200035b0 con 0x7f00380ce830 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698954311 0 0) 0x7f00200035b0 con 0x7f00380ce830 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 msgr2=0x7f00380ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f00380ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 >> v1:172.21.15.172:6789/0 conn(0x7f00380c6c30 legacy=0x7f00380cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380d03e0 con 0x7f00380ce830 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0028001670 con 0x7f00380ce830 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 >> v1:172.21.15.114:6789/0 conn(0x7f00380ce830 legacy=0x7f00380cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0020003a30 0x7f0020003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020004850 0x7f0020004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f0020005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003815c330 con 0x7f0020003a30 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002000bac0 con 0x7f0020004850 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002000bc80 con 0x7f003815bf50 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f0036d9d700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0020003a30 0x7f0020003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f003659c700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020004850 0x7f0020004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.062+0000 7f003759e700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f0020005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 -- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 msgr2=0x7f0020005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f003815bf50 0x7f0020005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 -- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020004850 msgr2=0x7f0020004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020004850 0x7f0020004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 -- 172.21.15.114:0/1730454197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200035b0 con 0x7f0020003a30 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0036d9d700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0020003a30 0x7f0020003e50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f002c009b30 tx=0x7f0020005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4522f6a16050fb0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00e6c0 con 0x7f0020003a30 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00e820 con 0x7f0020003a30 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f0035d9b700 1 -- 172.21.15.114:0/1730454197 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00eb90 con 0x7f0020003a30 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 -- 172.21.15.114:0/1730454197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0020003a30 msgr2=0x7f0020003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 --2- 172.21.15.114:0/1730454197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0020003a30 0x7f0020003e50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f002c009b30 tx=0x7f0020005880 comp rx=0 tx=0).stop 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003759e700 1 -- 172.21.15.114:0/1730454197 reap_dead start 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 -- 172.21.15.114:0/1730454197 shutdown_connections 2023-11-24T06:25:36.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 -- 172.21.15.114:0/1730454197 >> 172.21.15.114:0/1730454197 conn(0x7f00380b3780 msgr2=0x7f00380b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 -- 172.21.15.114:0/1730454197 shutdown_connections 2023-11-24T06:25:36.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.063+0000 7f003d159700 1 -- 172.21.15.114:0/1730454197 wait complete. 2023-11-24T06:25:36.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.065+0000 7f003d159700 1 Processor -- start 2023-11-24T06:25:36.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.065+0000 7f003d159700 1 -- start start 2023-11-24T06:25:36.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020003a30 0x7f00380b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00380cb6f0 0x7f00380b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00380b4870 con 0x7f00380b74d0 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00380b4a80 con 0x7f0020003a30 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00380b4d60 con 0x7f00380cb6f0 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40366/0 (socket says 172.21.15.114:40366) 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 -- 172.21.15.114:0/3771569612 learned_addr learned my addr 172.21.15.114:0/3771569612 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:36.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f0036d9d700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020003a30 0x7f00380b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003659c700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00380cb6f0 0x7f00380b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 -- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00380cb6f0 msgr2=0x7f00380b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00380cb6f0 0x7f00380b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 -- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020003a30 msgr2=0x7f00380b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020003a30 0x7f00380b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 -- 172.21.15.114:0/3771569612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380b5040 con 0x7f00380b74d0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003759e700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f80 tx=0x7f003000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030003c10 con 0x7f00380b74d0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0030003d70 con 0x7f00380b74d0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00380b5320 con 0x7f00380b74d0 2023-11-24T06:25:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030005310 con 0x7f00380b74d0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.066+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00380cbb70 con 0x7f00380b74d0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.067+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00380cbd60 con 0x7f00380b74d0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.068+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0030005470 con 0x7f00380b74d0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.068+0000 7f001f7fe700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 0x7f002406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.068+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00300897a0 con 0x7f00380b74d0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.068+0000 7f0036d9d700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 0x7f002406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.068+0000 7f0036d9d700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 0x7f002406d540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f002c0090b0 tx=0x7f002c01f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.072+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0030054660 con 0x7f00380b74d0 2023-11-24T06:25:36.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.264+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00380b5510 con 0x7f00380b74d0 2023-11-24T06:25:36.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.264+0000 7f001f7fe700 1 -- 172.21.15.114:0/3771569612 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0030057c80 con 0x7f00380b74d0 2023-11-24T06:25:36.267 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:25:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.266+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 msgr2=0x7f002406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.266+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 0x7f002406d540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f002c0090b0 tx=0x7f002c01f040 comp rx=0 tx=0).stop 2023-11-24T06:25:36.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 msgr2=0x7f00381e5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f80 tx=0x7f003000edd0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 shutdown_connections 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f002406b080 0x7f002406d540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00380b74d0 0x7f00381e5590 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020003a30 0x7f00380b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 --2- 172.21.15.114:0/3771569612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00380cb6f0 0x7f00380b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 >> 172.21.15.114:0/3771569612 conn(0x7f00380b3780 msgr2=0x7f003806ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 shutdown_connections 2023-11-24T06:25:36.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.267+0000 7f003d159700 1 -- 172.21.15.114:0/3771569612 wait complete. 2023-11-24T06:25:36.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:36.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.399+0000 7f847e09b700 1 Processor -- start 2023-11-24T06:25:36.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 -- start start 2023-11-24T06:25:36.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84780b59f0 0x7f847806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 0x7f84780b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780b0290 con 0x7f84780b5dd0 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84780b4820 con 0x7f84780b59f0 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f847e09b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780b4b00 con 0x7f847806e0b0 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.400+0000 7f84777fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84780b59f0 0x7f847806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f84780b59f0 0x7f847806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40370/0 (socket says 172.21.15.114:40370) 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 -- 172.21.15.114:0/4190770286 learned_addr learned my addr 172.21.15.114:0/4190770286 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:36.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f8476ffd700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 0x7f84780b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 -- 172.21.15.114:0/4190770286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 msgr2=0x7f84780b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 0x7f84780b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 -- 172.21.15.114:0/4190770286 >> v1:172.21.15.172:6789/0 conn(0x7f84780b5dd0 legacy=0x7f84780af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 -- 172.21.15.114:0/4190770286 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780ae9b0 con 0x7f84780b59f0 2023-11-24T06:25:36.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f8476ffd700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 0x7f84780b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:36.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84777fe700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84780b59f0 0x7f847806db70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f84680098f0 tx=0x7f8468009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658706aa7172dd2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84767fc700 1 -- 172.21.15.114:0/4190770286 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468004230 con 0x7f84780b59f0 2023-11-24T06:25:36.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84767fc700 1 -- 172.21.15.114:0/4190770286 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8468004390 con 0x7f84780b59f0 2023-11-24T06:25:36.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.401+0000 7f84767fc700 1 -- 172.21.15.114:0/4190770286 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84680036a0 con 0x7f84780b59f0 2023-11-24T06:25:36.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 -- 172.21.15.114:0/4190770286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84780b59f0 msgr2=0x7f847806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84780b59f0 0x7f847806db70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f84680098f0 tx=0x7f8468009c00 comp rx=0 tx=0).stop 2023-11-24T06:25:36.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 -- 172.21.15.114:0/4190770286 shutdown_connections 2023-11-24T06:25:36.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84780b59f0 0x7f847806db70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 --2- 172.21.15.114:0/4190770286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f847806e0b0 0x7f84780b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 -- 172.21.15.114:0/4190770286 >> 172.21.15.114:0/4190770286 conn(0x7f8478072990 msgr2=0x7f847806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 -- 172.21.15.114:0/4190770286 shutdown_connections 2023-11-24T06:25:36.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.402+0000 7f847e09b700 1 -- 172.21.15.114:0/4190770286 wait complete. 2023-11-24T06:25:36.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.403+0000 7f847e09b700 1 Processor -- start 2023-11-24T06:25:36.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.403+0000 7f847e09b700 1 -- start start 2023-11-24T06:25:36.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 0x7f8478155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84780b5dd0 0x7f84781555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478150f10 con 0x7f847806e0b0 2023-11-24T06:25:36.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478153ca0 con 0x7f84780b59f0 2023-11-24T06:25:36.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8478150640 con 0x7f84780b5dd0 2023-11-24T06:25:36.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40376/0 (socket says 172.21.15.114:40376) 2023-11-24T06:25:36.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 -- 172.21.15.114:0/281616030 learned_addr learned my addr 172.21.15.114:0/281616030 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:36.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f8476ffd700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 0x7f8478155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f8477fff700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84780b5dd0 0x7f84781555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 -- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84780b5dd0 msgr2=0x7f84781555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84780b5dd0 0x7f84781555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 -- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 msgr2=0x7f8478155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 0x7f8478155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 -- 172.21.15.114:0/281616030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009550 con 0x7f847806e0b0 2023-11-24T06:25:36.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f8476ffd700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 0x7f8478155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:36.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f84777fe700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f846800b0e0 tx=0x7f8468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468003e30 con 0x7f847806e0b0 2023-11-24T06:25:36.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846801f070 con 0x7f847806e0b0 2023-11-24T06:25:36.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.404+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8478155af0 con 0x7f847806e0b0 2023-11-24T06:25:36.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.405+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800b8a0 con 0x7f847806e0b0 2023-11-24T06:25:36.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.405+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8478150b30 con 0x7f847806e0b0 2023-11-24T06:25:36.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.405+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8478067f60 con 0x7f847806e0b0 2023-11-24T06:25:36.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.406+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8468021030 con 0x7f847806e0b0 2023-11-24T06:25:36.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.406+0000 7f846ffff700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 0x7f845806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.407+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f846808af50 con 0x7f847806e0b0 2023-11-24T06:25:36.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.407+0000 7f8476ffd700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 0x7f845806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.407+0000 7f8476ffd700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 0x7f845806d540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84780d0140 tx=0x7f8460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.411+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8468055e10 con 0x7f847806e0b0 2023-11-24T06:25:36.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.647+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8478067f60 con 0x7f847806e0b0 2023-11-24T06:25:36.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.648+0000 7f846ffff700 1 -- 172.21.15.114:0/281616030 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f8468059430 con 0x7f847806e0b0 2023-11-24T06:25:36.650 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:36.650 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 msgr2=0x7f845806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 0x7f845806d540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84780d0140 tx=0x7f8460008040 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 msgr2=0x7f8478144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f846800b0e0 tx=0x7f8468004750 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 shutdown_connections 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f845806b080 0x7f845806d540 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84780b5dd0 0x7f84781555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f847806e0b0 0x7f8478144380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 --2- 172.21.15.114:0/281616030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84780b59f0 0x7f8478155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 >> 172.21.15.114:0/281616030 conn(0x7f8478072990 msgr2=0x7f847806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 shutdown_connections 2023-11-24T06:25:36.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.650+0000 7f847e09b700 1 -- 172.21.15.114:0/281616030 wait complete. 2023-11-24T06:25:36.654 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2023-11-24T06:25:36.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:36.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 Processor -- start 2023-11-24T06:25:36.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 -- start start 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f98300adf00 0x7f983006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98300af950 con 0x7f98301446a0 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98300afc30 con 0x7f983006c540 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f983658f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983006c9f0 con 0x7f98300adf00 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f982ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f98301446a0 0x7f98300b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56092/0 (socket says 172.21.15.114:56092) 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.783+0000 7f982ffff700 1 -- 172.21.15.114:0/2878488628 learned_addr learned my addr 172.21.15.114:0/2878488628 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:36.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012714790 0 0) 0x7f98300af950 con 0x7f98301446a0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f981c0035b0 con 0x7f98301446a0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 --2- 172.21.15.114:0/2878488628 >> v2:172.21.15.172:3300/0 conn(0x7f98300adf00 0x7f983006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4020968100 0 0) 0x7f981c0035b0 con 0x7f98301446a0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 >> v2:172.21.15.172:3300/0 conn(0x7f98300adf00 msgr2=0x7f983006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 --2- 172.21.15.114:0/2878488628 >> v2:172.21.15.172:3300/0 conn(0x7f98300adf00 0x7f983006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 >> v1:172.21.15.172:6789/0 conn(0x7f983006c540 legacy=0x7f98300b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300aff10 con 0x7f98301446a0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f9820001670 con 0x7f98301446a0 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 >> v1:172.21.15.114:6789/0 conn(0x7f98301446a0 legacy=0x7f98300b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f981c003a00 0x7f981c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 0x7f981c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c005210 0x7f981c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983006c9f0 con 0x7f981c004850 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00c0b0 con 0x7f981c003a00 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00c270 con 0x7f981c005210 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 0x7f981c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982ffff700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f981c003a00 0x7f981c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f9834b2c700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c005210 0x7f981c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 -- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c005210 msgr2=0x7f981c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c005210 0x7f981c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 -- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f981c003a00 msgr2=0x7f981c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f981c003a00 0x7f981c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f982f7fe700 1 -- 172.21.15.114:0/2878488628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0035b0 con 0x7f981c004850 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.784+0000 7f9834b2c700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c005210 0x7f981c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f9834b2c700 1 -- 172.21.15.114:0/2878488628 reap_dead start 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f982f7fe700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 0x7f981c004cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f983006eef0 tx=0x7f981c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1eb1c033d85880 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800e690 con 0x7f981c004850 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981800e7f0 con 0x7f981c004850 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f982effd700 1 -- 172.21.15.114:0/2878488628 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800eb60 con 0x7f981c004850 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 -- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 msgr2=0x7f981c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 0x7f981c004cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f983006eef0 tx=0x7f981c005880 comp rx=0 tx=0).stop 2023-11-24T06:25:36.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 -- 172.21.15.114:0/2878488628 shutdown_connections 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 --2- 172.21.15.114:0/2878488628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c004850 0x7f981c004cd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 -- 172.21.15.114:0/2878488628 >> 172.21.15.114:0/2878488628 conn(0x7f9830071b90 msgr2=0x7f9830071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 -- 172.21.15.114:0/2878488628 shutdown_connections 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.785+0000 7f983658f700 1 -- 172.21.15.114:0/2878488628 wait complete. 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.787+0000 7f983658f700 1 Processor -- start 2023-11-24T06:25:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.787+0000 7f983658f700 1 -- start start 2023-11-24T06:25:36.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.787+0000 7f983658f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c004850 0x7f9830073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.787+0000 7f983658f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.787+0000 7f983658f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98301510f0 0x7f983014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f983658f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830151af0 con 0x7f98300744d0 2023-11-24T06:25:36.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f983658f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830074950 con 0x7f98301510f0 2023-11-24T06:25:36.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f983658f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983014bd90 con 0x7f981c004850 2023-11-24T06:25:36.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40398/0 (socket says 172.21.15.114:40398) 2023-11-24T06:25:36.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 -- 172.21.15.114:0/3615221080 learned_addr learned my addr 172.21.15.114:0/3615221080 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:36.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f9834b2c700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98301510f0 0x7f983014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982ffff700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c004850 0x7f9830073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 -- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c004850 msgr2=0x7f9830073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c004850 0x7f9830073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 -- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98301510f0 msgr2=0x7f983014d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98301510f0 0x7f983014d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 -- 172.21.15.114:0/3615221080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830151570 con 0x7f98300744d0 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982f7fe700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9818000c00 tx=0x7f98180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800e690 con 0x7f98300744d0 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180038a0 con 0x7f98300744d0 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818006e60 con 0x7f98300744d0 2023-11-24T06:25:36.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983014e340 con 0x7f98300744d0 2023-11-24T06:25:36.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.788+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983014b740 con 0x7f98300744d0 2023-11-24T06:25:36.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.789+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98300753f0 con 0x7f98300744d0 2023-11-24T06:25:36.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.790+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9818003c50 con 0x7f98300744d0 2023-11-24T06:25:36.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.790+0000 7f982cff9700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 0x7f981006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:36.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.790+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f981808afb0 con 0x7f98300744d0 2023-11-24T06:25:36.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.790+0000 7f982ffff700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 0x7f981006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:36.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.790+0000 7f982ffff700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 0x7f981006d540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9820001050 tx=0x7f9820000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:36.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.794+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98180bde90 con 0x7f98300744d0 2023-11-24T06:25:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.988+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9830142e10 con 0x7f98300744d0 2023-11-24T06:25:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.989+0000 7f982cff9700 1 -- 172.21.15.114:0/3615221080 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f9818017070 con 0x7f98300744d0 2023-11-24T06:25:36.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:36.992 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 msgr2=0x7f981006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 0x7f981006d540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9820001050 tx=0x7f9820000f80 comp rx=0 tx=0).stop 2023-11-24T06:25:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 msgr2=0x7f9830150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9818000c00 tx=0x7f98180047c0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 shutdown_connections 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f981006b080 0x7f981006d540 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98301510f0 0x7f983014d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f981c004850 0x7f9830073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.991+0000 7f983658f700 1 --2- 172.21.15.114:0/3615221080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98300744d0 0x7f9830150bb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.992+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 >> 172.21.15.114:0/3615221080 conn(0x7f9830071b90 msgr2=0x7f9830071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.992+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 shutdown_connections 2023-11-24T06:25:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:36.992+0000 7f983658f700 1 -- 172.21.15.114:0/3615221080 wait complete. 2023-11-24T06:25:37.006 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:25:37.006 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.124+0000 7f2a65a90700 1 Processor -- start 2023-11-24T06:25:37.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.124+0000 7f2a65a90700 1 -- start start 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a65a90700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a6015f180 0x7f2a600cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a65a90700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600b4290 con 0x7f2a6013e000 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a65a90700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6013e570 con 0x7f2a600cfd40 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a65a90700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600d05d0 con 0x7f2a6015f180 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5effd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a6015f180 0x7f2a600cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5f7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2a6013e000 0x7f2a6015ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56112/0 (socket says 172.21.15.114:56112) 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5f7fe700 1 -- 172.21.15.114:0/1430000292 learned_addr learned my addr 172.21.15.114:0/1430000292 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:37.129 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5effd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2a6015f180 0x7f2a600cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40406/0 (socket says 172.21.15.114:40406) 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5effd700 1 -- 172.21.15.114:0/1430000292 >> v1:172.21.15.172:6789/0 conn(0x7f2a600cfd40 legacy=0x7f2a600b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5effd700 1 -- 172.21.15.114:0/1430000292 >> v1:172.21.15.114:6789/0 conn(0x7f2a6013e000 legacy=0x7f2a6015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5effd700 1 -- 172.21.15.114:0/1430000292 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600d08b0 con 0x7f2a6015f180 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.125+0000 7f2a5e7fc700 1 -- 172.21.15.114:0/1430000292 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608343154 0 0) 0x7f2a600b4290 con 0x7f2a6013e000 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a5effd700 1 --2- 172.21.15.114:0/1430000292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6015f180 0x7f2a600cf800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009840 tx=0x7f2a50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a6153c32e1276f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a5e7fc700 1 -- 172.21.15.114:0/1430000292 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011830 con 0x7f2a6015f180 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a5e7fc700 1 -- 172.21.15.114:0/1430000292 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50011990 con 0x7f2a6015f180 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a5e7fc700 1 -- 172.21.15.114:0/1430000292 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011d20 con 0x7f2a6015f180 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 -- 172.21.15.114:0/1430000292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6015f180 msgr2=0x7f2a600cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1430000292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6015f180 0x7f2a600cf800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009840 tx=0x7f2a50009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 -- 172.21.15.114:0/1430000292 shutdown_connections 2023-11-24T06:25:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1430000292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6015f180 0x7f2a600cf800 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 -- 172.21.15.114:0/1430000292 >> 172.21.15.114:0/1430000292 conn(0x7f2a60072990 msgr2=0x7f2a6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 -- 172.21.15.114:0/1430000292 shutdown_connections 2023-11-24T06:25:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.126+0000 7f2a65a90700 1 -- 172.21.15.114:0/1430000292 wait complete. 2023-11-24T06:25:37.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.128+0000 7f2a65a90700 1 Processor -- start 2023-11-24T06:25:37.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.128+0000 7f2a65a90700 1 -- start start 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.128+0000 7f2a65a90700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600cfd40 0x7f2a6012d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.128+0000 7f2a65a90700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a6015f180 0x7f2a60143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601f50a0 con 0x7f2a6013e000 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601f52b0 con 0x7f2a600cfd40 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a601f5590 con 0x7f2a6015f180 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40412/0 (socket says 172.21.15.114:40412) 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 -- 172.21.15.114:0/1235240538 learned_addr learned my addr 172.21.15.114:0/1235240538 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5f7fe700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600cfd40 0x7f2a6012d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5ffff700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a6015f180 0x7f2a60143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 -- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a6015f180 msgr2=0x7f2a60143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a6015f180 0x7f2a60143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 -- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600cfd40 msgr2=0x7f2a6012d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600cfd40 0x7f2a6012d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 -- 172.21.15.114:0/1235240538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500094a0 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a5effd700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a50014040 tx=0x7f2a50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001a7e0 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5001a940 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001acb0 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a601f5870 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.129+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a601f5e30 con 0x7f2a6013e000 2023-11-24T06:25:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.130+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60067f60 con 0x7f2a6013e000 2023-11-24T06:25:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.131+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2a50022070 con 0x7f2a6013e000 2023-11-24T06:25:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.132+0000 7f2a47fff700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 0x7f2a4c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.132+0000 7f2a5f7fe700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 0x7f2a4c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.132+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a5008aa30 con 0x7f2a6013e000 2023-11-24T06:25:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.132+0000 7f2a5f7fe700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 0x7f2a4c06d540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a54007b10 tx=0x7f2a54007a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.136+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a50055ba0 con 0x7f2a6013e000 2023-11-24T06:25:37.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.367+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a600b5e70 con 0x7f2a6013e000 2023-11-24T06:25:37.370 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.368+0000 7f2a47fff700 1 -- 172.21.15.114:0/1235240538 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f2a500591c0 con 0x7f2a6013e000 2023-11-24T06:25:37.370 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:37.370 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:37.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 msgr2=0x7f2a4c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.372 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 0x7f2a4c06d540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a54007b10 tx=0x7f2a54007a60 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 msgr2=0x7f2a6015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a50014040 tx=0x7f2a50004750 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 shutdown_connections 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2a4c06b080 0x7f2a4c06d540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a6015f180 0x7f2a60143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a600cfd40 0x7f2a6012d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 --2- 172.21.15.114:0/1235240538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a6013e000 0x7f2a6015a2d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 >> 172.21.15.114:0/1235240538 conn(0x7f2a60072990 msgr2=0x7f2a600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 shutdown_connections 2023-11-24T06:25:37.373 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.370+0000 7f2a65a90700 1 -- 172.21.15.114:0/1235240538 wait complete. 2023-11-24T06:25:37.374 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2023-11-24T06:25:37.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.501+0000 7f96cc9a2700 1 Processor -- start 2023-11-24T06:25:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.501+0000 7f96cc9a2700 1 -- start start 2023-11-24T06:25:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.502+0000 7f96cc9a2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 0x7f96c80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.502+0000 7f96cc9a2700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96c80af4d0 0x7f96c80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.502+0000 7f96cc9a2700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c80ae430 con 0x7f96c806c000 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.502+0000 7f96cc9a2700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c80af0b0 con 0x7f96c80af4d0 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.502+0000 7f96cc9a2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80b05a0 con 0x7f96c815e080 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96c80af4d0 0x7f96c80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c6d9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f96c806c000 0x7f96c8073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56130/0 (socket says 172.21.15.114:56130) 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c6d9d700 1 -- 172.21.15.114:0/3016814528 learned_addr learned my addr 172.21.15.114:0/3016814528 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c659c700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 0x7f96c80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 -- 172.21.15.114:0/3016814528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 msgr2=0x7f96c80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 0x7f96c80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 -- 172.21.15.114:0/3016814528 >> v1:172.21.15.114:6789/0 conn(0x7f96c806c000 legacy=0x7f96c8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 -- 172.21.15.114:0/3016814528 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80ae710 con 0x7f96c80af4d0 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c559a700 1 -- 172.21.15.114:0/3016814528 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020886264 0 0) 0x7f96c80ae430 con 0x7f96c806c000 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c659c700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 0x7f96c80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c5d9b700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c80af4d0 0x7f96c80afb90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009870 tx=0x7f96b8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee6fc7701f736ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c559a700 1 -- 172.21.15.114:0/3016814528 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011830 con 0x7f96c80af4d0 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c559a700 1 -- 172.21.15.114:0/3016814528 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8011990 con 0x7f96c80af4d0 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.503+0000 7f96c559a700 1 -- 172.21.15.114:0/3016814528 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8011d20 con 0x7f96c80af4d0 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 -- 172.21.15.114:0/3016814528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c80af4d0 msgr2=0x7f96c80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c80af4d0 0x7f96c80afb90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009870 tx=0x7f96b8009b80 comp rx=0 tx=0).stop 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 -- 172.21.15.114:0/3016814528 shutdown_connections 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c815e080 0x7f96c80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/3016814528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c80af4d0 0x7f96c80afb90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 -- 172.21.15.114:0/3016814528 >> 172.21.15.114:0/3016814528 conn(0x7f96c8071b90 msgr2=0x7f96c8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 -- 172.21.15.114:0/3016814528 shutdown_connections 2023-11-24T06:25:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.504+0000 7f96cc9a2700 1 -- 172.21.15.114:0/3016814528 wait complete. 2023-11-24T06:25:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.505+0000 7f96cc9a2700 1 Processor -- start 2023-11-24T06:25:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.505+0000 7f96cc9a2700 1 -- start start 2023-11-24T06:25:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96c806c000 0x7f96c80ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 0x7f96c8160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80cf940 con 0x7f96c815e080 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80ced30 con 0x7f96c806c000 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96cc9a2700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80cf010 con 0x7f96c80af4d0 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96c6d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96c6d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40426/0 (socket says 172.21.15.114:40426) 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96c6d9d700 1 -- 172.21.15.114:0/578007083 learned_addr learned my addr 172.21.15.114:0/578007083 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96c659c700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96c806c000 0x7f96c80ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.506+0000 7f96c5d9b700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 0x7f96c8160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 -- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 msgr2=0x7f96c8160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 0x7f96c8160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 -- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96c806c000 msgr2=0x7f96c80ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96c806c000 0x7f96c80ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 -- 172.21.15.114:0/578007083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80094a0 con 0x7f96c815e080 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c5d9b700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 0x7f96c8160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96c6d9d700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f96c0002230 tx=0x7f96c000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000ab70 con 0x7f96c815e080 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c000acf0 con 0x7f96c815e080 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0007310 con 0x7f96c815e080 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c80cf2f0 con 0x7f96c815e080 2023-11-24T06:25:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.507+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c8161360 con 0x7f96c815e080 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.508+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c80753f0 con 0x7f96c815e080 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.508+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f96c0007470 con 0x7f96c815e080 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.509+0000 7f96b6ffd700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 0x7f96a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.509+0000 7f96c659c700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 0x7f96a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.509+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96c0089360 con 0x7f96c815e080 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.509+0000 7f96c659c700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 0x7f96a806d540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00c940 tx=0x7f96bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.513+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96c0054220 con 0x7f96c815e080 2023-11-24T06:25:37.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.743+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96c8067f60 con 0x7f96c815e080 2023-11-24T06:25:37.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.743+0000 7f96b6ffd700 1 -- 172.21.15.114:0/578007083 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f96c000ae60 con 0x7f96c815e080 2023-11-24T06:25:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:37.746 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:17.870023+0000","modified":"2023-11-24T06:25:20.896679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1960617883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":1960617883},{"type":"v1","addr":"172.21.15.114:6837","nonce":1960617883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:25:37.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 msgr2=0x7f96a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 0x7f96a806d540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00c940 tx=0x7f96bc008040 comp rx=0 tx=0).stop 2023-11-24T06:25:37.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 msgr2=0x7f96c81445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f96c0002230 tx=0x7f96c000cf50 comp rx=0 tx=0).stop 2023-11-24T06:25:37.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 shutdown_connections 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f96a806b080 0x7f96a806d540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96c815e080 0x7f96c81445f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96c806c000 0x7f96c80ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.745+0000 7f96cc9a2700 1 --2- 172.21.15.114:0/578007083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96c80af4d0 0x7f96c8160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.746+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 >> 172.21.15.114:0/578007083 conn(0x7f96c8071b90 msgr2=0x7f96c80b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.746+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 shutdown_connections 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.746+0000 7f96cc9a2700 1 -- 172.21.15.114:0/578007083 wait complete. 2023-11-24T06:25:37.749 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2023-11-24T06:25:37.760 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:37.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.878+0000 7fe0cbcf3700 1 Processor -- start 2023-11-24T06:25:37.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.878+0000 7fe0cbcf3700 1 -- start start 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0cbcf3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe0c40d0780 0x7fe0c40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0cbcf3700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40cd5d0 con 0x7fe0c40ae910 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0cbcf3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40cd870 con 0x7fe0c415c030 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0cbcf3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40cdb50 con 0x7fe0c40d0780 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0c928e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe0c40d0780 0x7fe0c40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0ca290700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe0c40ae910 0x7fe0c40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56144/0 (socket says 172.21.15.114:56144) 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.879+0000 7fe0ca290700 1 -- 172.21.15.114:0/3837479795 learned_addr learned my addr 172.21.15.114:0/3837479795 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c928e700 1 -- 172.21.15.114:0/3837479795 >> v1:172.21.15.172:6789/0 conn(0x7fe0c415c030 legacy=0x7fe0c40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c928e700 1 -- 172.21.15.114:0/3837479795 >> v1:172.21.15.114:6789/0 conn(0x7fe0c40ae910 legacy=0x7fe0c40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c8a8d700 1 -- 172.21.15.114:0/3837479795 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942679326 0 0) 0x7fe0c40cd5d0 con 0x7fe0c40ae910 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c928e700 1 -- 172.21.15.114:0/3837479795 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40cde30 con 0x7fe0c40d0780 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c928e700 1 --2- 172.21.15.114:0/3837479795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c40cc8c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009840 tx=0x7fe0b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df724c0213f8189 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c8a8d700 1 -- 172.21.15.114:0/3837479795 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011830 con 0x7fe0c40d0780 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c8a8d700 1 -- 172.21.15.114:0/3837479795 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4011990 con 0x7fe0c40d0780 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0c8a8d700 1 -- 172.21.15.114:0/3837479795 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011d20 con 0x7fe0c40d0780 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/3837479795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 msgr2=0x7fe0c40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/3837479795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c40cc8c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009840 tx=0x7fe0b4009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/3837479795 shutdown_connections 2023-11-24T06:25:37.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.880+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/3837479795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c40cc8c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.881+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/3837479795 >> 172.21.15.114:0/3837479795 conn(0x7fe0c40730a0 msgr2=0x7fe0c409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:37.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.881+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/3837479795 shutdown_connections 2023-11-24T06:25:37.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.881+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/3837479795 wait complete. 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 Processor -- start 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 -- start start 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0c40ae910 0x7fe0c40ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0c40b08d0 0x7fe0c4155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c5ff0 con 0x7fe0c40d0780 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40cb1a0 con 0x7fe0c40b08d0 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.882+0000 7fe0cbcf3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40caec0 con 0x7fe0c40ae910 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40440/0 (socket says 172.21.15.114:40440) 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 -- 172.21.15.114:0/1874887416 learned_addr learned my addr 172.21.15.114:0/1874887416 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0ca290700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0c40b08d0 0x7fe0c4155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c9a8f700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0c40ae910 0x7fe0c40ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 -- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0c40ae910 msgr2=0x7fe0c40ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0c40ae910 0x7fe0c40ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 -- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0c40b08d0 msgr2=0x7fe0c4155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0c40b08d0 0x7fe0c4155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 -- 172.21.15.114:0/1874887416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40094a0 con 0x7fe0c40d0780 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0c928e700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40090b0 tx=0x7fe0b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40045d0 con 0x7fe0c40d0780 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c40c59a0 con 0x7fe0c40d0780 2023-11-24T06:25:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c414da70 con 0x7fe0c40d0780 2023-11-24T06:25:37.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b401f070 con 0x7fe0c40d0780 2023-11-24T06:25:37.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.883+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40038a0 con 0x7fe0c40d0780 2023-11-24T06:25:37.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.884+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac0052e0 con 0x7fe0c40d0780 2023-11-24T06:25:37.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.885+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe0b4003a00 con 0x7fe0c40d0780 2023-11-24T06:25:37.889 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.885+0000 7fe0ba7fc700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 0x7fe0a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:37.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.885+0000 7fe0c9a8f700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 0x7fe0a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:37.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.885+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b4021070 con 0x7fe0c40d0780 2023-11-24T06:25:37.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.886+0000 7fe0c9a8f700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 0x7fe0a806d540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007fd0 tx=0x7fe0c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:37.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:37.890+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0b4056030 con 0x7fe0c40d0780 2023-11-24T06:25:38.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.080+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0ac005860 con 0x7fe0c40d0780 2023-11-24T06:25:38.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.082+0000 7fe0ba7fc700 1 -- 172.21.15.114:0/1874887416 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fe0b4003cb0 con 0x7fe0c40d0780 2023-11-24T06:25:38.085 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:38.085 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:19.491342+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:15.659348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:16.837675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:38.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 msgr2=0x7fe0a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 0x7fe0a806d540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007fd0 tx=0x7fe0c0009040 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 msgr2=0x7fe0c415b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40090b0 tx=0x7fe0b4004750 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 shutdown_connections 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a806b080 0x7fe0a806d540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0c40b08d0 0x7fe0c4155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0c40ae910 0x7fe0c40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 --2- 172.21.15.114:0/1874887416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0c40d0780 0x7fe0c415b3c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 >> 172.21.15.114:0/1874887416 conn(0x7fe0c40730a0 msgr2=0x7fe0c40cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:38.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 shutdown_connections 2023-11-24T06:25:38.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.084+0000 7fe0cbcf3700 1 -- 172.21.15.114:0/1874887416 wait complete. 2023-11-24T06:25:38.099 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:25:38.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.217+0000 7f985982d700 1 Processor -- start 2023-11-24T06:25:38.220 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.217+0000 7f985982d700 1 -- start start 2023-11-24T06:25:38.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f985982d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9854152cc0 0x7f985406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f985982d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540d0630 0x7f985406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f985982d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985415c4b0 con 0x7f98540b49a0 2023-11-24T06:25:38.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f985982d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985415c790 con 0x7f9854152cc0 2023-11-24T06:25:38.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f985982d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985415ca70 con 0x7f98540d0630 2023-11-24T06:25:38.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f9852ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9854152cc0 0x7f985406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f98527fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f98540b49a0 0x7f98540cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56154/0 (socket says 172.21.15.114:56154) 2023-11-24T06:25:38.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.218+0000 7f98527fc700 1 -- 172.21.15.114:0/1784927568 learned_addr learned my addr 172.21.15.114:0/1784927568 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:38.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f98537fe700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540d0630 0x7f985406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9852ffd700 1 -- 172.21.15.114:0/1784927568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540d0630 msgr2=0x7f985406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9852ffd700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540d0630 0x7f985406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9852ffd700 1 -- 172.21.15.114:0/1784927568 >> v1:172.21.15.114:6789/0 conn(0x7f98540b49a0 legacy=0x7f98540cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9852ffd700 1 -- 172.21.15.114:0/1784927568 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985415cd50 con 0x7f9854152cc0 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9852ffd700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9854152cc0 0x7f985406c000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9848009840 tx=0x7f9848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc2d1942b90a724 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9851ffb700 1 -- 172.21.15.114:0/1784927568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011830 con 0x7f9854152cc0 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9851ffb700 1 -- 172.21.15.114:0/1784927568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848011990 con 0x7f9854152cc0 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f9851ffb700 1 -- 172.21.15.114:0/1784927568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848011d20 con 0x7f9854152cc0 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f985982d700 1 -- 172.21.15.114:0/1784927568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9854152cc0 msgr2=0x7f985406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.219+0000 7f985982d700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9854152cc0 0x7f985406c000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9848009840 tx=0x7f9848009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 -- 172.21.15.114:0/1784927568 shutdown_connections 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540d0630 0x7f985406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 --2- 172.21.15.114:0/1784927568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9854152cc0 0x7f985406c000 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 -- 172.21.15.114:0/1784927568 >> 172.21.15.114:0/1784927568 conn(0x7f9854071b90 msgr2=0x7f9854071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 -- 172.21.15.114:0/1784927568 shutdown_connections 2023-11-24T06:25:38.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.220+0000 7f985982d700 1 -- 172.21.15.114:0/1784927568 wait complete. 2023-11-24T06:25:38.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.221+0000 7f985982d700 1 Processor -- start 2023-11-24T06:25:38.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.221+0000 7f985982d700 1 -- start start 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 0x7f9854150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 0x7f9854154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854155de0 con 0x7f98540d0630 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541577a0 con 0x7f9854152cc0 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985415f100 con 0x7f98540b49a0 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.230 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40452/0 (socket says 172.21.15.114:40452) 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 -- 172.21.15.114:0/672139155 learned_addr learned my addr 172.21.15.114:0/672139155 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98537fe700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 0x7f9854154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f9852ffd700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 0x7f9854150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 -- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 msgr2=0x7f9854150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 0x7f9854150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 -- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 msgr2=0x7f9854154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 0x7f9854154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 -- 172.21.15.114:0/672139155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480094a0 con 0x7f98540d0630 2023-11-24T06:25:38.231 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f9852ffd700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 0x7f9854150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98537fe700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 0x7f9854154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f98527fc700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f984400c520 tx=0x7f984400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844005d70 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98440070d0 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854155b00 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844007440 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.222+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854157d00 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.223+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98540753f0 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.224+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f98440075a0 con 0x7f98540d0630 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.224+0000 7f983bfff700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 0x7f984006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.232 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.224+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9844088dc0 con 0x7f98540d0630 2023-11-24T06:25:38.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.224+0000 7f9852ffd700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 0x7f984006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.225+0000 7f9852ffd700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 0x7f984006d570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98480090b0 tx=0x7f9848008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.229+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9844053b70 con 0x7f98540d0630 2023-11-24T06:25:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.464+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9854067f60 con 0x7f98540d0630 2023-11-24T06:25:38.467 INFO:tasks.ceph.mon.a.smithi114.stderr:2023-11-24T06:25:38.464+0000 7f8fd8393700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:25:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.466+0000 7f983bfff700 1 -- 172.21.15.114:0/672139155 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9844017020 con 0x7f98540d0630 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 msgr2=0x7f984006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 0x7f984006d570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98480090b0 tx=0x7f9848008030 comp rx=0 tx=0).stop 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 msgr2=0x7f9854158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f984400c520 tx=0x7f984400c8e0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 shutdown_connections 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f984006b0b0 0x7f984006d570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9854152cc0 0x7f9854154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98540b49a0 0x7f9854150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 --2- 172.21.15.114:0/672139155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98540d0630 0x7f9854158500 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 >> 172.21.15.114:0/672139155 conn(0x7f9854071b90 msgr2=0x7f985406ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.469+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 shutdown_connections 2023-11-24T06:25:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.470+0000 7f985982d700 1 -- 172.21.15.114:0/672139155 wait complete. 2023-11-24T06:25:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:25:38.484 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:25:38.509 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:25:38.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.606+0000 7f93a7513700 1 Processor -- start 2023-11-24T06:25:38.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.606+0000 7f93a7513700 1 -- start start 2023-11-24T06:25:38.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a7513700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93a015ac40 0x7f93a00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a7513700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a7513700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00b2a60 con 0x7f93a006e930 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a7513700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a015e970 con 0x7f93a015ac40 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a7513700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015ef30 con 0x7f93a00d05b0 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:35018/0 (socket says 172.21.15.114:35018) 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 -- 172.21.15.114:0/4202731188 learned_addr learned my addr 172.21.15.114:0/4202731188 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a52af700 1 --2- 172.21.15.114:0/4202731188 >> v2:172.21.15.172:3300/0 conn(0x7f93a015ac40 0x7f93a00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 -- 172.21.15.114:0/4202731188 >> v2:172.21.15.172:3300/0 conn(0x7f93a015ac40 msgr2=0x7f93a00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:38.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 --2- 172.21.15.114:0/4202731188 >> v2:172.21.15.172:3300/0 conn(0x7f93a015ac40 0x7f93a00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 -- 172.21.15.114:0/4202731188 >> v1:172.21.15.172:6789/0 conn(0x7f93a006e930 legacy=0x7f93a00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.607+0000 7f93a5ab0700 1 -- 172.21.15.114:0/4202731188 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a015f900 con 0x7f93a00d05b0 2023-11-24T06:25:38.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a5ab0700 1 --2- 172.21.15.114:0/4202731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f939800d150 tx=0x7f939800d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8529609aaa67504b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f9397fff700 1 -- 172.21.15.114:0/4202731188 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398010070 con 0x7f93a00d05b0 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f9397fff700 1 -- 172.21.15.114:0/4202731188 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939800dd10 con 0x7f93a00d05b0 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f9397fff700 1 -- 172.21.15.114:0/4202731188 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800b310 con 0x7f93a00d05b0 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 -- 172.21.15.114:0/4202731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 msgr2=0x7f93a015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 --2- 172.21.15.114:0/4202731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f939800d150 tx=0x7f939800d460 comp rx=0 tx=0).stop 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 -- 172.21.15.114:0/4202731188 shutdown_connections 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 --2- 172.21.15.114:0/4202731188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a00d05b0 0x7f93a015f3c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 --2- 172.21.15.114:0/4202731188 >> v2:172.21.15.172:3300/0 conn(0x7f93a015ac40 0x7f93a00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 -- 172.21.15.114:0/4202731188 >> 172.21.15.114:0/4202731188 conn(0x7f93a006b410 msgr2=0x7f93a006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 -- 172.21.15.114:0/4202731188 shutdown_connections 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.608+0000 7f93a7513700 1 -- 172.21.15.114:0/4202731188 wait complete. 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 Processor -- start 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 -- start start 2023-11-24T06:25:38.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a006e930 0x7f93a0140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a015ac40 0x7f93a013e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a013f640 con 0x7f93a00d05b0 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0140010 con 0x7f93a015ac40 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a7513700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a013e780 con 0x7f93a006e930 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a4aae700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a4aae700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40478/0 (socket says 172.21.15.114:40478) 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a4aae700 1 -- 172.21.15.114:0/2975195744 learned_addr learned my addr 172.21.15.114:0/2975195744 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a52af700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a006e930 0x7f93a0140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.610+0000 7f93a5ab0700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a015ac40 0x7f93a013e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 -- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a006e930 msgr2=0x7f93a0140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a006e930 0x7f93a0140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 -- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a015ac40 msgr2=0x7f93a013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a015ac40 0x7f93a013e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 -- 172.21.15.114:0/2975195744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939800bd40 con 0x7f93a00d05b0 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a4aae700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f939000c4e0 tx=0x7f939000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900052d0 con 0x7f93a00d05b0 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390005430 con 0x7f93a00d05b0 2023-11-24T06:25:38.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a013ea30 con 0x7f93a00d05b0 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900038c0 con 0x7f93a00d05b0 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.611+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a013eff0 con 0x7f93a00d05b0 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.612+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0068480 con 0x7f93a00d05b0 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.612+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9390003a20 con 0x7f93a00d05b0 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.613+0000 7f9395ffb700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 0x7f938406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:38.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.613+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9390088cc0 con 0x7f93a00d05b0 2023-11-24T06:25:38.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.613+0000 7f93a52af700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 0x7f938406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:38.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.613+0000 7f93a52af700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 0x7f938406d540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93a006e080 tx=0x7f939c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:38.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.617+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9390053a90 con 0x7f93a00d05b0 2023-11-24T06:25:38.854 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:38.852+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f93a0068fb0 con 0x7f93a00d05b0 2023-11-24T06:25:39.470 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.468+0000 7f9395ffb700 1 -- 172.21.15.114:0/2975195744 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f93900570b0 con 0x7f93a00d05b0 2023-11-24T06:25:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 msgr2=0x7f938406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 0x7f938406d540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93a006e080 tx=0x7f939c009210 comp rx=0 tx=0).stop 2023-11-24T06:25:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 msgr2=0x7f93a013fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f939000c4e0 tx=0x7f939000c7f0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 shutdown_connections 2023-11-24T06:25:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f938406b080 0x7f938406d540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a015ac40 0x7f93a013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93a006e930 0x7f93a0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 --2- 172.21.15.114:0/2975195744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a00d05b0 0x7f93a013fad0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 >> 172.21.15.114:0/2975195744 conn(0x7f93a006b410 msgr2=0x7f93a0072260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 shutdown_connections 2023-11-24T06:25:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.470+0000 7f93a7513700 1 -- 172.21.15.114:0/2975195744 wait complete. 2023-11-24T06:25:39.487 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:25:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.652+0000 7fc625289700 1 Processor -- start 2023-11-24T06:25:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.652+0000 7fc625289700 1 -- start start 2023-11-24T06:25:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc625289700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6200b01e0 0x7fc62006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc625289700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc6200746a0 0x7fc6200b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc625289700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6200b05c0 con 0x7fc6200b59c0 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc625289700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6201470a0 con 0x7fc6200746a0 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc625289700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6200b5e70 con 0x7fc6200b01e0 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc61f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc6200746a0 0x7fc6200b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc61f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc6200746a0 0x7fc6200b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40490/0 (socket says 172.21.15.114:40490) 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc61f7fe700 1 -- 172.21.15.114:0/834555111 learned_addr learned my addr 172.21.15.114:0/834555111 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.653+0000 7fc61e7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc6200b59c0 0x7fc620073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56194/0 (socket says 172.21.15.114:56194) 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61effd700 1 --2- 172.21.15.114:0/834555111 >> v2:172.21.15.172:3300/0 conn(0x7fc6200b01e0 0x7fc62006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61f7fe700 1 -- 172.21.15.114:0/834555111 >> v2:172.21.15.172:3300/0 conn(0x7fc6200b01e0 msgr2=0x7fc62006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61f7fe700 1 --2- 172.21.15.114:0/834555111 >> v2:172.21.15.172:3300/0 conn(0x7fc6200b01e0 0x7fc62006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61f7fe700 1 -- 172.21.15.114:0/834555111 >> v1:172.21.15.114:6789/0 conn(0x7fc6200b59c0 legacy=0x7fc620073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61f7fe700 1 -- 172.21.15.114:0/834555111 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200b4250 con 0x7fc6200746a0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61dffb700 1 -- 172.21.15.114:0/834555111 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859101409 0 0) 0x7fc6200b05c0 con 0x7fc6200b59c0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61f7fe700 1 --2- 172.21.15.114:0/834555111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200746a0 0x7fc6200b3b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc620072ce0 tx=0x7fc61800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4ee45e2dacdb1b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61dffb700 1 -- 172.21.15.114:0/834555111 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800e070 con 0x7fc6200746a0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61dffb700 1 -- 172.21.15.114:0/834555111 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61800b980 con 0x7fc6200746a0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.654+0000 7fc61dffb700 1 -- 172.21.15.114:0/834555111 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61800bd30 con 0x7fc6200746a0 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 -- 172.21.15.114:0/834555111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200746a0 msgr2=0x7fc6200b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 --2- 172.21.15.114:0/834555111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200746a0 0x7fc6200b3b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc620072ce0 tx=0x7fc61800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 -- 172.21.15.114:0/834555111 shutdown_connections 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 --2- 172.21.15.114:0/834555111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200746a0 0x7fc6200b3b50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 --2- 172.21.15.114:0/834555111 >> v2:172.21.15.172:3300/0 conn(0x7fc6200b01e0 0x7fc62006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 -- 172.21.15.114:0/834555111 >> 172.21.15.114:0/834555111 conn(0x7fc6200d0b40 msgr2=0x7fc6200d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 -- 172.21.15.114:0/834555111 shutdown_connections 2023-11-24T06:25:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.655+0000 7fc625289700 1 -- 172.21.15.114:0/834555111 wait complete. 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.656+0000 7fc625289700 1 Processor -- start 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.656+0000 7fc625289700 1 -- start start 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6200746a0 0x7fc6201f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6200b59c0 0x7fc6200aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201f6e90 con 0x7fc6200b01e0 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201f40a0 con 0x7fc6200b59c0 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6201f4380 con 0x7fc6200746a0 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40494/0 (socket says 172.21.15.114:40494) 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 -- 172.21.15.114:0/4253335098 learned_addr learned my addr 172.21.15.114:0/4253335098 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61f7fe700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6200b59c0 0x7fc6200aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61effd700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6200746a0 0x7fc6201f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 -- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6200746a0 msgr2=0x7fc6201f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6200746a0 0x7fc6201f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 -- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6200b59c0 msgr2=0x7fc6200aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6200b59c0 0x7fc6200aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 -- 172.21.15.114:0/4253335098 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618009b30 con 0x7fc6200b01e0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc61e7fc700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c520 tx=0x7fc61000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610004c80 con 0x7fc6200b01e0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc610004e20 con 0x7fc6200b01e0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6201f7100 con 0x7fc6200b01e0 2023-11-24T06:25:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610007310 con 0x7fc6200b01e0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.657+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6201f4660 con 0x7fc6200b01e0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.658+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc620067f60 con 0x7fc6200b01e0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.659+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc610007470 con 0x7fc6200b01e0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.659+0000 7fc6077fe700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 0x7fc60c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.659+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc610089650 con 0x7fc6200b01e0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.659+0000 7fc61effd700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 0x7fc60c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:39.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.660+0000 7fc61effd700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 0x7fc60c06d540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc614009930 tx=0x7fc614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:39.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.664+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc610054430 con 0x7fc6200b01e0 2023-11-24T06:25:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:39.855+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc62006c980 con 0x7fc6200b01e0 2023-11-24T06:25:40.475 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.472+0000 7fc6077fe700 1 -- 172.21.15.114:0/4253335098 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc610057a50 con 0x7fc6200b01e0 2023-11-24T06:25:40.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 msgr2=0x7fc60c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 0x7fc60c06d540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc614009930 tx=0x7fc614008040 comp rx=0 tx=0).stop 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 msgr2=0x7fc6201f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c520 tx=0x7fc61000c8e0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 shutdown_connections 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc60c06b080 0x7fc60c06d540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6200b59c0 0x7fc6200aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.479 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6200746a0 0x7fc6201f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 --2- 172.21.15.114:0/4253335098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6200b01e0 0x7fc6201f7620 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.475+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 >> 172.21.15.114:0/4253335098 conn(0x7fc6200d0b40 msgr2=0x7fc620061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:40.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.476+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 shutdown_connections 2023-11-24T06:25:40.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.476+0000 7fc625289700 1 -- 172.21.15.114:0/4253335098 wait complete. 2023-11-24T06:25:40.480 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:25:40.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:25:40.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.617+0000 7f35553c4700 1 Processor -- start 2023-11-24T06:25:40.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.617+0000 7f35553c4700 1 -- start start 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f35553c4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f355015dfe0 0x7f35500d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f35553c4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f35500d39a0 0x7f35500b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f35553c4700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35500d1e60 con 0x7f355015e3c0 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f35553c4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500d2ae0 con 0x7f355015dfe0 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f35553c4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35500d2140 con 0x7f35500d39a0 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354effd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f355015dfe0 0x7f35500d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354f7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f355015e3c0 0x7f355006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56212/0 (socket says 172.21.15.114:56212) 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354f7fe700 1 -- 172.21.15.114:0/241724763 learned_addr learned my addr 172.21.15.114:0/241724763 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354e7fc700 1 --2- 172.21.15.114:0/241724763 >> v2:172.21.15.172:3300/0 conn(0x7f35500d39a0 0x7f35500b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354effd700 1 -- 172.21.15.114:0/241724763 >> v2:172.21.15.172:3300/0 conn(0x7f35500d39a0 msgr2=0x7f35500b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354effd700 1 --2- 172.21.15.114:0/241724763 >> v2:172.21.15.172:3300/0 conn(0x7f35500d39a0 0x7f35500b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354effd700 1 -- 172.21.15.114:0/241724763 >> v1:172.21.15.114:6789/0 conn(0x7f355015e3c0 legacy=0x7f355006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354effd700 1 -- 172.21.15.114:0/241724763 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500d2420 con 0x7f355015dfe0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.618+0000 7f354dffb700 1 -- 172.21.15.114:0/241724763 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158505636 0 0) 0x7f35500d1e60 con 0x7f355015e3c0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f354effd700 1 --2- 172.21.15.114:0/241724763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500d3460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35500b6d90 tx=0x7f3544009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c476469550ead7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f354dffb700 1 -- 172.21.15.114:0/241724763 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544011830 con 0x7f355015dfe0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f354dffb700 1 -- 172.21.15.114:0/241724763 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3544011990 con 0x7f355015dfe0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f354dffb700 1 -- 172.21.15.114:0/241724763 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544011d20 con 0x7f355015dfe0 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 -- 172.21.15.114:0/241724763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 msgr2=0x7f35500d3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 --2- 172.21.15.114:0/241724763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500d3460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35500b6d90 tx=0x7f3544009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 -- 172.21.15.114:0/241724763 shutdown_connections 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 --2- 172.21.15.114:0/241724763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500d3460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 --2- 172.21.15.114:0/241724763 >> v2:172.21.15.172:3300/0 conn(0x7f35500d39a0 0x7f35500b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 -- 172.21.15.114:0/241724763 >> 172.21.15.114:0/241724763 conn(0x7f35500d0b10 msgr2=0x7f35500d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 -- 172.21.15.114:0/241724763 shutdown_connections 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.619+0000 7f35553c4700 1 -- 172.21.15.114:0/241724763 wait complete. 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 Processor -- start 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 -- start start 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35500d39a0 0x7f35500b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 0x7f35500cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550160710 con 0x7f355015dfe0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355014de70 con 0x7f355015e3c0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.621+0000 7f35553c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355014d4d0 con 0x7f35500d39a0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40508/0 (socket says 172.21.15.114:40508) 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 -- 172.21.15.114:0/1971223355 learned_addr learned my addr 172.21.15.114:0/1971223355 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354f7fe700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 0x7f35500cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354effd700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35500d39a0 0x7f35500b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 -- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35500d39a0 msgr2=0x7f35500b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35500d39a0 0x7f35500b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 -- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 msgr2=0x7f35500cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 0x7f35500cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 -- 172.21.15.114:0/1971223355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35440094a0 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354f7fe700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 0x7f35500cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f354e7fc700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f354000c7f0 tx=0x7f354000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540011800 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3540011960 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355014d7b0 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540011cf0 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.622+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35500b2790 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.623+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355014dfb0 con 0x7f355015dfe0 2023-11-24T06:25:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.623+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3540018360 con 0x7f355015dfe0 2023-11-24T06:25:40.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.624+0000 7f35377fe700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 0x7f353c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:40.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.624+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f3540089790 con 0x7f355015dfe0 2023-11-24T06:25:40.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.624+0000 7f354effd700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 0x7f353c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:40.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.624+0000 7f354effd700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 0x7f353c06d540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f3544008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:40.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.628+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f355014dfb0 con 0x7f355015dfe0 2023-11-24T06:25:40.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:40.818+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f355014dfb0 con 0x7f355015dfe0 2023-11-24T06:25:41.476 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.474+0000 7f35377fe700 1 -- 172.21.15.114:0/1971223355 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f355014dfb0 con 0x7f355015dfe0 2023-11-24T06:25:41.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 msgr2=0x7f353c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 0x7f353c06d540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3544000c00 tx=0x7f3544008030 comp rx=0 tx=0).stop 2023-11-24T06:25:41.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 msgr2=0x7f35500cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f354000c7f0 tx=0x7f354000cbb0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.480 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 shutdown_connections 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f353c06b080 0x7f353c06d540 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355015e3c0 0x7f35500cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35500d39a0 0x7f35500b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 --2- 172.21.15.114:0/1971223355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f355015dfe0 0x7f35500cd740 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 >> 172.21.15.114:0/1971223355 conn(0x7f35500d0b10 msgr2=0x7f35500b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.477+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 shutdown_connections 2023-11-24T06:25:41.481 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.478+0000 7f35553c4700 1 -- 172.21.15.114:0/1971223355 wait complete. 2023-11-24T06:25:41.482 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2023-11-24T06:25:41.495 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:25:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.616+0000 7f6f36587700 1 Processor -- start 2023-11-24T06:25:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.616+0000 7f6f36587700 1 -- start start 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f36587700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f300b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f36587700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f300b7860 0x7f6f300cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f36587700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3006c210 con 0x7f6f3015dfa0 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f36587700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f300b4b30 con 0x7f6f300b7860 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f36587700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300b4e10 con 0x7f6f300b6ec0 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f2ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6f3015dfa0 0x7f6f3006ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56222/0 (socket says 172.21.15.114:56222) 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.617+0000 7f6f2ffff700 1 -- 172.21.15.114:0/1250116772 learned_addr learned my addr 172.21.15.114:0/1250116772 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943257931 0 0) 0x7f6f3006c210 con 0x7f6f3015dfa0 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f180035b0 con 0x7f6f3015dfa0 2023-11-24T06:25:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f34b24700 1 --2- 172.21.15.114:0/1250116772 >> v2:172.21.15.172:3300/0 conn(0x7f6f300b7860 0x7f6f300cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f300b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669650008 0 0) 0x7f6f180035b0 con 0x7f6f3015dfa0 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 msgr2=0x7f6f300b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f300b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 >> v2:172.21.15.172:3300/0 conn(0x7f6f300b7860 msgr2=0x7f6f300cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 --2- 172.21.15.114:0/1250116772 >> v2:172.21.15.172:3300/0 conn(0x7f6f300b7860 0x7f6f300cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300b50f0 con 0x7f6f3015dfa0 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f6f24001670 con 0x7f6f3015dfa0 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 >> v1:172.21.15.114:6789/0 conn(0x7f6f3015dfa0 legacy=0x7f6f3006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f18003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f18004030 0x7f6f18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004d90 0x7f6f18005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300b4b30 con 0x7f6f18004030 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300b4e10 con 0x7f6f18004d90 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1800bb10 con 0x7f6f300b6ec0 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f18004030 0x7f6f18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2ffff700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f18003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f34b24700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004d90 0x7f6f18005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 -- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 msgr2=0x7f6f18003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f300b6ec0 0x7f6f18003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 -- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004d90 msgr2=0x7f6f18005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004d90 0x7f6f18005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 -- 172.21.15.114:0/1250116772 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180035b0 con 0x7f6f18004030 2023-11-24T06:25:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.618+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f18004030 0x7f6f18004850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009890 tx=0x7f6f18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4a0921c4137af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000c070 con 0x7f6f18004030 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f20013610 con 0x7f6f18004030 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f2effd700 1 -- 172.21.15.114:0/1250116772 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200139a0 con 0x7f6f18004030 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 -- 172.21.15.114:0/1250116772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f18004030 msgr2=0x7f6f18004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 --2- 172.21.15.114:0/1250116772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f18004030 0x7f6f18004850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009890 tx=0x7f6f18005880 comp rx=0 tx=0).stop 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f34b24700 1 -- 172.21.15.114:0/1250116772 reap_dead start 2023-11-24T06:25:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 -- 172.21.15.114:0/1250116772 shutdown_connections 2023-11-24T06:25:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 -- 172.21.15.114:0/1250116772 >> 172.21.15.114:0/1250116772 conn(0x7f6f300b3810 msgr2=0x7f6f300b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 -- 172.21.15.114:0/1250116772 shutdown_connections 2023-11-24T06:25:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.619+0000 7f6f36587700 1 -- 172.21.15.114:0/1250116772 wait complete. 2023-11-24T06:25:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.620+0000 7f6f36587700 1 Processor -- start 2023-11-24T06:25:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 -- start start 2023-11-24T06:25:41.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004030 0x7f6f300c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 0x7f6f3015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300c9610 con 0x7f6f3015f2d0 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3015ff40 con 0x7f6f18004030 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.621+0000 7f6f36587700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3015fc60 con 0x7f6f3015ca30 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40530/0 (socket says 172.21.15.114:40530) 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 -- 172.21.15.114:0/993762892 learned_addr learned my addr 172.21.15.114:0/993762892 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2ffff700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004030 0x7f6f300c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 0x7f6f3015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 -- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 msgr2=0x7f6f3015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 0x7f6f3015c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 -- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004030 msgr2=0x7f6f300c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004030 0x7f6f300c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 -- 172.21.15.114:0/993762892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300c8f50 con 0x7f6f3015f2d0 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2f7fe700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 0x7f6f3015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f34b24700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6f300b4450 tx=0x7f6f280048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28007900 con 0x7f6f3015f2d0 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f28007a60 con 0x7f6f3015f2d0 2023-11-24T06:25:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f300c9230 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28007e10 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.622+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3015f750 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.623+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f300c8db0 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.624+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6f28015420 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.624+0000 7f6f2cff9700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 0x7f6f1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.624+0000 7f6f2ffff700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 0x7f6f1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.624+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f280899c0 con 0x7f6f3015f2d0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.624+0000 7f6f2ffff700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 0x7f6f1c06d540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c00 tx=0x7f6f24007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.629+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f300c8db0 con 0x7f6f3015f2d0 2023-11-24T06:25:41.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.830+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6f300c8db0 con 0x7f6f3015f2d0 2023-11-24T06:25:41.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.832+0000 7f6f2cff9700 1 -- 172.21.15.114:0/993762892 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f300c8db0 con 0x7f6f3015f2d0 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 msgr2=0x7f6f1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 0x7f6f1c06d540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c00 tx=0x7f6f24007980 comp rx=0 tx=0).stop 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 msgr2=0x7f6f300b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6f300b4450 tx=0x7f6f280048c0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 shutdown_connections 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f1c06b080 0x7f6f1c06d540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3015f2d0 0x7f6f300b60c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f18004030 0x7f6f300c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 --2- 172.21.15.114:0/993762892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f3015ca30 0x7f6f3015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 >> 172.21.15.114:0/993762892 conn(0x7f6f300b3810 msgr2=0x7f6f3006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:41.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 shutdown_connections 2023-11-24T06:25:41.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.835+0000 7f6f36587700 1 -- 172.21.15.114:0/993762892 wait complete. 2023-11-24T06:25:41.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-24T06:25:41.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.971+0000 7f9841a93700 1 Processor -- start 2023-11-24T06:25:41.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.971+0000 7f9841a93700 1 -- start start 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f9841a93700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f983c15e080 0x7f983c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f9841a93700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f983c06c710 0x7f983c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f9841a93700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c0ae430 con 0x7f983c0af4d0 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f9841a93700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c0af0b0 con 0x7f983c15e080 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f9841a93700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c0b05a0 con 0x7f983c06c710 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f983c15e080 0x7f983c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983affd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f983c0af4d0 0x7f983c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56242/0 (socket says 172.21.15.114:56242) 2023-11-24T06:25:41.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983affd700 1 -- 172.21.15.114:0/256970908 learned_addr learned my addr 172.21.15.114:0/256970908 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983bfff700 1 --2- 172.21.15.114:0/256970908 >> v2:172.21.15.172:3300/0 conn(0x7f983c06c710 0x7f983c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983b7fe700 1 -- 172.21.15.114:0/256970908 >> v2:172.21.15.172:3300/0 conn(0x7f983c06c710 msgr2=0x7f983c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983b7fe700 1 --2- 172.21.15.114:0/256970908 >> v2:172.21.15.172:3300/0 conn(0x7f983c06c710 0x7f983c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983b7fe700 1 -- 172.21.15.114:0/256970908 >> v1:172.21.15.114:6789/0 conn(0x7f983c0af4d0 legacy=0x7f983c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983a7fc700 1 -- 172.21.15.114:0/256970908 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125479160 0 0) 0x7f983c0ae430 con 0x7f983c0af4d0 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.972+0000 7f983b7fe700 1 -- 172.21.15.114:0/256970908 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0ae710 con 0x7f983c15e080 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f983b7fe700 1 --2- 172.21.15.114:0/256970908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c15e080 0x7f983c0b5050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f983c06e060 tx=0x7f9834009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39888374fcbb9d22 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f983a7fc700 1 -- 172.21.15.114:0/256970908 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834011830 con 0x7f983c15e080 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f983a7fc700 1 -- 172.21.15.114:0/256970908 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9834011990 con 0x7f983c15e080 2023-11-24T06:25:41.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f983a7fc700 1 -- 172.21.15.114:0/256970908 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834011d20 con 0x7f983c15e080 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 -- 172.21.15.114:0/256970908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c15e080 msgr2=0x7f983c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 --2- 172.21.15.114:0/256970908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c15e080 0x7f983c0b5050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f983c06e060 tx=0x7f9834009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 -- 172.21.15.114:0/256970908 shutdown_connections 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 --2- 172.21.15.114:0/256970908 >> v2:172.21.15.172:3300/0 conn(0x7f983c06c710 0x7f983c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 --2- 172.21.15.114:0/256970908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c15e080 0x7f983c0b5050 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 -- 172.21.15.114:0/256970908 >> 172.21.15.114:0/256970908 conn(0x7f983c071b90 msgr2=0x7f983c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 -- 172.21.15.114:0/256970908 shutdown_connections 2023-11-24T06:25:41.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.973+0000 7f9841a93700 1 -- 172.21.15.114:0/256970908 wait complete. 2023-11-24T06:25:41.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.974+0000 7f9841a93700 1 Processor -- start 2023-11-24T06:25:41.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 -- start start 2023-11-24T06:25:41.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f983c06c710 0x7f983c160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 0x7f983c15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c1592f0 con 0x7f983c0af4d0 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15b800 con 0x7f983c06c710 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.975+0000 7f9841a93700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c1589c0 con 0x7f983c1423a0 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40542/0 (socket says 172.21.15.114:40542) 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 -- 172.21.15.114:0/382331598 learned_addr learned my addr 172.21.15.114:0/382331598 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983b7fe700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f983c06c710 0x7f983c160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983bfff700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 0x7f983c15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 -- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 msgr2=0x7f983c15aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 0x7f983c15aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 -- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f983c06c710 msgr2=0x7f983c160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f983c06c710 0x7f983c160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 -- 172.21.15.114:0/382331598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340094a0 con 0x7f983c0af4d0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983bfff700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 0x7f983c15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f983affd700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f983000c520 tx=0x7f983000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830004c80 con 0x7f983c0af4d0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9830004de0 con 0x7f983c0af4d0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300072d0 con 0x7f983c0af4d0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c158ca0 con 0x7f983c0af4d0 2023-11-24T06:25:41.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.976+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c15afd0 con 0x7f983c0af4d0 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.977+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c0753f0 con 0x7f983c0af4d0 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.978+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9830007430 con 0x7f983c0af4d0 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.978+0000 7f9823fff700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 0x7f982806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.978+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9830089220 con 0x7f983c0af4d0 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.978+0000 7f983b7fe700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 0x7f982806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:41.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.978+0000 7f983b7fe700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 0x7f982806d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f983c06d350 tx=0x7f9834008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:41.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:41.983+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98300543e0 con 0x7f983c0af4d0 2023-11-24T06:25:42.262 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.260+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f983c067f60 con 0x7f983c0af4d0 2023-11-24T06:25:42.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.481+0000 7f9823fff700 1 -- 172.21.15.114:0/382331598 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v102) v1 ==== 158+0+0 (secure 0 0 0) 0x7f9830057a00 con 0x7f983c0af4d0 2023-11-24T06:25:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 msgr2=0x7f982806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 0x7f982806d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f983c06d350 tx=0x7f9834008030 comp rx=0 tx=0).stop 2023-11-24T06:25:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 msgr2=0x7f983c160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f983000c520 tx=0x7f983000c830 comp rx=0 tx=0).stop 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 shutdown_connections 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f982806b080 0x7f982806d540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f983c1423a0 0x7f983c15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f983c06c710 0x7f983c160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 --2- 172.21.15.114:0/382331598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f983c0af4d0 0x7f983c160f50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 >> 172.21.15.114:0/382331598 conn(0x7f983c071b90 msgr2=0x7f983c0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 shutdown_connections 2023-11-24T06:25:42.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.483+0000 7f9841a93700 1 -- 172.21.15.114:0/382331598 wait complete. 2023-11-24T06:25:42.499 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-11-24T06:25:42.499 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:25:42.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:42.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.617+0000 7f6de4032700 1 Processor -- start 2023-11-24T06:25:42.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 -- start start 2023-11-24T06:25:42.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 0x7f6ddc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6ddc06c710 0x7f6ddc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc0ae430 con 0x7f6ddc0af4d0 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ddc0af0b0 con 0x7f6ddc06c710 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.618+0000 7f6de4032700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0b05a0 con 0x7f6ddc15e080 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6ddc06c710 0x7f6ddc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de15cd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6ddc0af4d0 0x7f6ddc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56258/0 (socket says 172.21.15.114:56258) 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de15cd700 1 -- 172.21.15.114:0/4198392999 learned_addr learned my addr 172.21.15.114:0/4198392999 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de1dce700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 0x7f6ddc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 -- 172.21.15.114:0/4198392999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 msgr2=0x7f6ddc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 0x7f6ddc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 -- 172.21.15.114:0/4198392999 >> v1:172.21.15.114:6789/0 conn(0x7f6ddc0af4d0 legacy=0x7f6ddc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 -- 172.21.15.114:0/4198392999 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0ae710 con 0x7f6ddc06c710 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de0dcc700 1 -- 172.21.15.114:0/4198392999 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249930996 0 0) 0x7f6ddc0ae430 con 0x7f6ddc0af4d0 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de1dce700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 0x7f6ddc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de25cf700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc06c710 0x7f6ddc0afb90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009dc0 tx=0x7f6dd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b6d8eeade2072 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de0dcc700 1 -- 172.21.15.114:0/4198392999 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400e070 con 0x7f6ddc06c710 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de0dcc700 1 -- 172.21.15.114:0/4198392999 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd400bb90 con 0x7f6ddc06c710 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.619+0000 7f6de0dcc700 1 -- 172.21.15.114:0/4198392999 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4012430 con 0x7f6ddc06c710 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 -- 172.21.15.114:0/4198392999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc06c710 msgr2=0x7f6ddc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc06c710 0x7f6ddc0afb90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009dc0 tx=0x7f6dd400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 -- 172.21.15.114:0/4198392999 shutdown_connections 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc15e080 0x7f6ddc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 --2- 172.21.15.114:0/4198392999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc06c710 0x7f6ddc0afb90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 -- 172.21.15.114:0/4198392999 >> 172.21.15.114:0/4198392999 conn(0x7f6ddc071b90 msgr2=0x7f6ddc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:42.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 -- 172.21.15.114:0/4198392999 shutdown_connections 2023-11-24T06:25:42.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.620+0000 7f6de4032700 1 -- 172.21.15.114:0/4198392999 wait complete. 2023-11-24T06:25:42.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.621+0000 7f6de4032700 1 Processor -- start 2023-11-24T06:25:42.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.621+0000 7f6de4032700 1 -- start start 2023-11-24T06:25:42.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ddc06c710 0x7f6ddc14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc0af4d0 0x7f6ddc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0c5bd0 con 0x7f6ddc15e080 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0c6470 con 0x7f6ddc06c710 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc14db90 con 0x7f6ddc0af4d0 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40558/0 (socket says 172.21.15.114:40558) 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 -- 172.21.15.114:0/905227842 learned_addr learned my addr 172.21.15.114:0/905227842 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de15cd700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc0af4d0 0x7f6ddc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de1dce700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ddc06c710 0x7f6ddc14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 -- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc0af4d0 msgr2=0x7f6ddc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc0af4d0 0x7f6ddc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 -- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ddc06c710 msgr2=0x7f6ddc14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ddc06c710 0x7f6ddc14d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 -- 172.21.15.114:0/905227842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4009a70 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de25cf700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc09f040 tx=0x7f6dd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4012430 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd4020070 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc158180 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.622+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004280 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.623+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc0c6100 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.623+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc0753f0 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.624+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6dd40043e0 con 0x7f6ddc15e080 2023-11-24T06:25:42.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.624+0000 7f6dd27fc700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 0x7f6dc006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.624+0000 7f6de1dce700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 0x7f6dc006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.624+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6dd4022070 con 0x7f6ddc15e080 2023-11-24T06:25:42.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.625+0000 7f6de1dce700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 0x7f6dc006d540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c900 tx=0x7f6dcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.629+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6dd4056d60 con 0x7f6ddc15e080 2023-11-24T06:25:42.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.821+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ddc068670 con 0x7f6ddc15e080 2023-11-24T06:25:42.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.821+0000 7f6dd27fc700 1 -- 172.21.15.114:0/905227842 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13024 (secure 0 0 0) 0x7f6dd405a380 con 0x7f6ddc15e080 2023-11-24T06:25:42.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:42.824 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:41.473092+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6837/1960617883":"2023-11-25T06:25:38.465159+0000","172.21.15.114:6835/1960617883":"2023-11-25T06:25:38.465159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:42.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.823+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 msgr2=0x7f6dc006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.823+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 0x7f6dc006d540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c900 tx=0x7f6dcc008040 comp rx=0 tx=0).stop 2023-11-24T06:25:42.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.823+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 msgr2=0x7f6ddc15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.823+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc09f040 tx=0x7f6dd4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 shutdown_connections 2023-11-24T06:25:42.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6dc006b080 0x7f6dc006d540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ddc15e080 0x7f6ddc15f9f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ddc06c710 0x7f6ddc14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 --2- 172.21.15.114:0/905227842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ddc0af4d0 0x7f6ddc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 >> 172.21.15.114:0/905227842 conn(0x7f6ddc071b90 msgr2=0x7f6ddc0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:42.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 shutdown_connections 2023-11-24T06:25:42.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.824+0000 7f6de4032700 1 -- 172.21.15.114:0/905227842 wait complete. 2023-11-24T06:25:42.838 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:42.839 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:42.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.959+0000 7f416ff89700 1 Processor -- start 2023-11-24T06:25:42.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.959+0000 7f416ff89700 1 -- start start 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416ff89700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41680746a0 0x7f41680b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416ff89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680afc90 0x7f416806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416ff89700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680b3c60 con 0x7f416815e160 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416ff89700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41680b4500 con 0x7f41680746a0 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416ff89700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680b58c0 con 0x7f41680afc90 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416d524700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41680746a0 0x7f41680b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416dd25700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f416815e160 0x7f4168073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56270/0 (socket says 172.21.15.114:56270) 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416dd25700 1 -- 172.21.15.114:0/2955584585 learned_addr learned my addr 172.21.15.114:0/2955584585 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:42.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416e526700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680afc90 0x7f416806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416d524700 1 -- 172.21.15.114:0/2955584585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680afc90 msgr2=0x7f416806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416d524700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680afc90 0x7f416806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416d524700 1 -- 172.21.15.114:0/2955584585 >> v1:172.21.15.114:6789/0 conn(0x7f416815e160 legacy=0x7f4168073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.960+0000 7f416d524700 1 -- 172.21.15.114:0/2955584585 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680b0120 con 0x7f41680746a0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416d524700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41680746a0 0x7f41680b5da0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4164009840 tx=0x7f4164009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1abdb51ad0ce86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416cd23700 1 -- 172.21.15.114:0/2955584585 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011830 con 0x7f41680746a0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416cd23700 1 -- 172.21.15.114:0/2955584585 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164011990 con 0x7f41680746a0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416cd23700 1 -- 172.21.15.114:0/2955584585 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011d20 con 0x7f41680746a0 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 -- 172.21.15.114:0/2955584585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41680746a0 msgr2=0x7f41680b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41680746a0 0x7f41680b5da0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4164009840 tx=0x7f4164009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 -- 172.21.15.114:0/2955584585 shutdown_connections 2023-11-24T06:25:42.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680afc90 0x7f416806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 --2- 172.21.15.114:0/2955584585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41680746a0 0x7f41680b5da0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 -- 172.21.15.114:0/2955584585 >> 172.21.15.114:0/2955584585 conn(0x7f41680d0bf0 msgr2=0x7f41680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:42.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 -- 172.21.15.114:0/2955584585 shutdown_connections 2023-11-24T06:25:42.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.961+0000 7f416ff89700 1 -- 172.21.15.114:0/2955584585 wait complete. 2023-11-24T06:25:42.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 Processor -- start 2023-11-24T06:25:42.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 -- start start 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680746a0 0x7f4168160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41680afc90 0x7f4168148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41680725e0 con 0x7f416815e160 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681608e0 con 0x7f41680afc90 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.963+0000 7f416ff89700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41681612b0 con 0x7f41680746a0 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40572/0 (socket says 172.21.15.114:40572) 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 -- 172.21.15.114:0/2865519104 learned_addr learned my addr 172.21.15.114:0/2865519104 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416dd25700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680746a0 0x7f4168160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416d524700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41680afc90 0x7f4168148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 -- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680746a0 msgr2=0x7f4168160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680746a0 0x7f4168160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 -- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41680afc90 msgr2=0x7f4168148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41680afc90 0x7f4168148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 -- 172.21.15.114:0/2865519104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640094a0 con 0x7f416815e160 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416e526700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4160007e70 tx=0x7f416000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000ed30 con 0x7f416815e160 2023-11-24T06:25:42.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000eeb0 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4168072b10 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160014640 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.964+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4168072830 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.965+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168062580 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.966+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f41600147a0 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.966+0000 7f415e7fc700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 0x7f414c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.966+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4160089cd0 con 0x7f416815e160 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.966+0000 7f416dd25700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 0x7f414c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:42.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.966+0000 7f416dd25700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 0x7f414c06d540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4158005910 tx=0x7f4158009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:42.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:42.970+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4160054e00 con 0x7f416815e160 2023-11-24T06:25:43.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.165+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41680b6160 con 0x7f416815e160 2023-11-24T06:25:43.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.165+0000 7f415e7fc700 1 -- 172.21.15.114:0/2865519104 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4160058420 con 0x7f416815e160 2023-11-24T06:25:43.169 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 msgr2=0x7f414c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 0x7f414c06d540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4158005910 tx=0x7f4158009040 comp rx=0 tx=0).stop 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 msgr2=0x7f4168155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4160007e70 tx=0x7f416000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 shutdown_connections 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f414c06b080 0x7f414c06d540 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f416815e160 0x7f4168155000 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f41680746a0 0x7f4168160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 --2- 172.21.15.114:0/2865519104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41680afc90 0x7f4168148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 >> 172.21.15.114:0/2865519104 conn(0x7f41680d0bf0 msgr2=0x7f416815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:43.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 shutdown_connections 2023-11-24T06:25:43.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.167+0000 7f416ff89700 1 -- 172.21.15.114:0/2865519104 wait complete. 2023-11-24T06:25:43.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-24T06:25:43.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.300+0000 7f86dd425700 1 Processor -- start 2023-11-24T06:25:43.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.300+0000 7f86dd425700 1 -- start start 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86dd425700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86d815e1e0 0x7f86d80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86dd425700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d80b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86dd425700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80d20b0 con 0x7f86d80d3a00 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86dd425700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80d3150 con 0x7f86d815e1e0 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86dd425700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815e5c0 con 0x7f86d80d0f80 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f86d815e1e0 0x7f86d80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:43.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d67fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f86d80d3a00 0x7f86d80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56284/0 (socket says 172.21.15.114:56284) 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d67fc700 1 -- 172.21.15.114:0/1589460760 learned_addr learned my addr 172.21.15.114:0/1589460760 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 -- 172.21.15.114:0/1589460760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 msgr2=0x7f86d80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 --2- 172.21.15.114:0/1589460760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d80b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 -- 172.21.15.114:0/1589460760 >> v1:172.21.15.114:6789/0 conn(0x7f86d80d3a00 legacy=0x7f86d80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d5ffb700 1 -- 172.21.15.114:0/1589460760 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307386197 0 0) 0x7f86d80d20b0 con 0x7f86d80d3a00 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 -- 172.21.15.114:0/1589460760 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80d2390 con 0x7f86d815e1e0 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.301+0000 7f86d6ffd700 1 --2- 172.21.15.114:0/1589460760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d80d34c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009870 tx=0x7f86cc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f7c0cdff60e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86d5ffb700 1 -- 172.21.15.114:0/1589460760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011830 con 0x7f86d815e1e0 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86d5ffb700 1 -- 172.21.15.114:0/1589460760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc011990 con 0x7f86d815e1e0 2023-11-24T06:25:43.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86d5ffb700 1 -- 172.21.15.114:0/1589460760 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011d20 con 0x7f86d815e1e0 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 -- 172.21.15.114:0/1589460760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 msgr2=0x7f86d80d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 --2- 172.21.15.114:0/1589460760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d80d34c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009870 tx=0x7f86cc009b80 comp rx=0 tx=0).stop 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 -- 172.21.15.114:0/1589460760 shutdown_connections 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 --2- 172.21.15.114:0/1589460760 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d80b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 --2- 172.21.15.114:0/1589460760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d80d34c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 -- 172.21.15.114:0/1589460760 >> 172.21.15.114:0/1589460760 conn(0x7f86d80b5b50 msgr2=0x7f86d80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 -- 172.21.15.114:0/1589460760 shutdown_connections 2023-11-24T06:25:43.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.302+0000 7f86dd425700 1 -- 172.21.15.114:0/1589460760 wait complete. 2023-11-24T06:25:43.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.303+0000 7f86dd425700 1 Processor -- start 2023-11-24T06:25:43.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 -- start start 2023-11-24T06:25:43.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d8159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 0x7f86d8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8159cd0 con 0x7f86d815e1e0 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815ab10 con 0x7f86d80d3a00 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86dd425700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8159f00 con 0x7f86d80d0f80 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86d77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86d77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40586/0 (socket says 172.21.15.114:40586) 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.304+0000 7f86d77fe700 1 -- 172.21.15.114:0/782766032 learned_addr learned my addr 172.21.15.114:0/782766032 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d6ffd700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d8159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:43.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d67fc700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 0x7f86d8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 -- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 msgr2=0x7f86d8159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d8159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 -- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 msgr2=0x7f86d8156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 0x7f86d8156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 -- 172.21.15.114:0/782766032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0094a0 con 0x7f86d815e1e0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d67fc700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 0x7f86d8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86d77fe700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b9e0 tx=0x7f86d000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000c9b0 con 0x7f86d815e1e0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d000cb30 con 0x7f86d815e1e0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d815a1e0 con 0x7f86d815e1e0 2023-11-24T06:25:43.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0014440 con 0x7f86d815e1e0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.305+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d815a7a0 con 0x7f86d815e1e0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.306+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d815acc0 con 0x7f86d815e1e0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.306+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f86d001d3b0 con 0x7f86d815e1e0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.307+0000 7f86bf7fe700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 0x7f86c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.307+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f86d0006e40 con 0x7f86d815e1e0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.307+0000 7f86d6ffd700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 0x7f86c406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:43.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.307+0000 7f86d6ffd700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 0x7f86c406d540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097c0 tx=0x7f86cc01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:43.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.311+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86d815acc0 con 0x7f86d815e1e0 2023-11-24T06:25:43.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:43.579+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f86d815ac80 con 0x7f86d815e1e0 2023-11-24T06:25:44.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.487+0000 7f86bf7fe700 1 -- 172.21.15.114:0/782766032 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v104) v1 ==== 143+0+0 (secure 0 0 0) 0x7f86d0087e60 con 0x7f86d815e1e0 2023-11-24T06:25:44.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.489+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 msgr2=0x7f86c406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.489+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 0x7f86c406d540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097c0 tx=0x7f86cc01c040 comp rx=0 tx=0).stop 2023-11-24T06:25:44.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.489+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 msgr2=0x7f86d8143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.489+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b9e0 tx=0x7f86d000bda0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.489+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 shutdown_connections 2023-11-24T06:25:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f86c406b080 0x7f86c406d540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86d815e1e0 0x7f86d8143130 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86d80d0f80 0x7f86d8159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 --2- 172.21.15.114:0/782766032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d80d3a00 0x7f86d8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 >> 172.21.15.114:0/782766032 conn(0x7f86d80b5b50 msgr2=0x7f86d815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 shutdown_connections 2023-11-24T06:25:44.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.490+0000 7f86dd425700 1 -- 172.21.15.114:0/782766032 wait complete. 2023-11-24T06:25:44.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:25:44.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.627+0000 7f469f1ca700 1 Processor -- start 2023-11-24T06:25:44.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.627+0000 7f469f1ca700 1 -- start start 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f469f1ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f469815e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f469f1ca700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980b2b00 con 0x7f4698151660 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f469f1ca700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469806e3a0 con 0x7f469806dec0 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f469f1ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469806ee40 con 0x7f4698070b40 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f4697fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4698151660 0x7f46980cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56298/0 (socket says 172.21.15.114:56298) 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f4697fff700 1 -- 172.21.15.114:0/1478064748 learned_addr learned my addr 172.21.15.114:0/1478064748 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024590510 0 0) 0x7f46980b2b00 con 0x7f4698151660 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f469d767700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f469815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.628+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46880035b0 con 0x7f4698151660 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630954792 0 0) 0x7f46880035b0 con 0x7f4698151660 2023-11-24T06:25:44.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 msgr2=0x7f469815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f469815e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 >> v1:172.21.15.172:6789/0 conn(0x7f469806dec0 legacy=0x7f46980cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980b2da0 con 0x7f4698151660 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f4684001670 con 0x7f4698151660 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 >> v1:172.21.15.114:6789/0 conn(0x7f4698151660 legacy=0x7f46980cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4688004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f4688005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806ee40 con 0x7f4688004850 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468800bb70 con 0x7f4688003a30 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468800bd30 con 0x7f4698070b40 2023-11-24T06:25:44.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4688004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f469cf66700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f469d767700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f4688005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 -- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 msgr2=0x7f4688005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f4688005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 -- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4688003a30 msgr2=0x7f4688003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 -- 172.21.15.114:0/1478064748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880035b0 con 0x7f4688004850 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f469d767700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698070b40 0x7f4688005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f469cf66700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f4697fff700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4688004cd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f468400c210 tx=0x7f4688005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7030c5ed63b798 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684007c10 con 0x7f4688004850 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468400cdb0 con 0x7f4688004850 2023-11-24T06:25:44.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.629+0000 7f46977fe700 1 -- 172.21.15.114:0/1478064748 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684005ba0 con 0x7f4688004850 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 -- 172.21.15.114:0/1478064748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 msgr2=0x7f4688004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 --2- 172.21.15.114:0/1478064748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4688004cd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f468400c210 tx=0x7f4688005880 comp rx=0 tx=0).stop 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469d767700 1 -- 172.21.15.114:0/1478064748 reap_dead start 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 -- 172.21.15.114:0/1478064748 shutdown_connections 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 -- 172.21.15.114:0/1478064748 >> 172.21.15.114:0/1478064748 conn(0x7f469806b410 msgr2=0x7f469806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 -- 172.21.15.114:0/1478064748 shutdown_connections 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.630+0000 7f469f1ca700 1 -- 172.21.15.114:0/1478064748 wait complete. 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.631+0000 7f469f1ca700 1 Processor -- start 2023-11-24T06:25:44.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.631+0000 7f469f1ca700 1 -- start start 2023-11-24T06:25:44.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 0x7f46980cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 0x7f4698159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469815a530 con 0x7f4688004850 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981556d0 con 0x7f469814f160 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469f1ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469814f5e0 con 0x7f4698159b70 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469cf66700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469cf66700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40602/0 (socket says 172.21.15.114:40602) 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469cf66700 1 -- 172.21.15.114:0/669307618 learned_addr learned my addr 172.21.15.114:0/669307618 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f469d767700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 0x7f4698159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.632+0000 7f4697fff700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 0x7f46980cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 -- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 msgr2=0x7f4698159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 0x7f4698159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 -- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 msgr2=0x7f46980cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 0x7f46980cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 -- 172.21.15.114:0/669307618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980cdb80 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f4697fff700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 0x7f46980cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469d767700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 0x7f4698159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469cf66700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f468c0109c0 tx=0x7f468c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0054e0 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c010ea0 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980cd000 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c006a80 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.633+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46980cd5c0 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.634+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698155990 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.634+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f468c006be0 con 0x7f4688004850 2023-11-24T06:25:44.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.635+0000 7f46957fa700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 0x7f467c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.635+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f468c08b900 con 0x7f4688004850 2023-11-24T06:25:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.635+0000 7f4697fff700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 0x7f467c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:44.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.635+0000 7f4697fff700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 0x7f467c06d540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46981f40e0 tx=0x7f4684007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:44.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.639+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f468c0569b0 con 0x7f4688004850 2023-11-24T06:25:44.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.830+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4698068480 con 0x7f4688004850 2023-11-24T06:25:44.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.834+0000 7f46957fa700 1 -- 172.21.15.114:0/669307618 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f468c01f0c0 con 0x7f4688004850 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 msgr2=0x7f467c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 0x7f467c06d540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46981f40e0 tx=0x7f4684007000 comp rx=0 tx=0).stop 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 msgr2=0x7f4698153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f468c0109c0 tx=0x7f468c010d80 comp rx=0 tx=0).stop 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 shutdown_connections 2023-11-24T06:25:44.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f467c06b080 0x7f467c06d540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4698159b70 0x7f4698159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4688004850 0x7f4698153da0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 --2- 172.21.15.114:0/669307618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f469814f160 0x7f46980cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 >> 172.21.15.114:0/669307618 conn(0x7f469806b410 msgr2=0x7f46980b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 shutdown_connections 2023-11-24T06:25:44.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:44.836+0000 7f469f1ca700 1 -- 172.21.15.114:0/669307618 wait complete. 2023-11-24T06:25:44.851 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:25:44.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:25:44.901 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:25:44.944 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:25:44.944 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:44.944 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:44.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:25:44.962 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:44.962 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:45.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:25:45.004 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:25:45.004 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:45.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:25:45.021 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:25:45.022 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:25:45.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:25:45.063 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:25:45.064 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:45.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.181+0000 7f92b3b94700 1 Processor -- start 2023-11-24T06:25:45.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.181+0000 7f92b3b94700 1 -- start start 2023-11-24T06:25:45.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b3b94700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92ac0af4d0 0x7f92ac0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b3b94700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0ae430 con 0x7f92ac06c710 2023-11-24T06:25:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b3b94700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0ae710 con 0x7f92ac15e080 2023-11-24T06:25:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b3b94700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac0af0b0 con 0x7f92ac0af4d0 2023-11-24T06:25:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b112f700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f92ac06c710 0x7f92ac06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56320/0 (socket says 172.21.15.114:56320) 2023-11-24T06:25:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b112f700 1 -- 172.21.15.114:0/3096707938 learned_addr learned my addr 172.21.15.114:0/3096707938 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b2131700 1 --2- 172.21.15.114:0/3096707938 >> v2:172.21.15.172:3300/0 conn(0x7f92ac0af4d0 0x7f92ac0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012220457 0 0) 0x7f92ac0ae430 con 0x7f92ac06c710 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92980035b0 con 0x7f92ac06c710 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977129763 0 0) 0x7f92980035b0 con 0x7f92ac06c710 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 >> v2:172.21.15.172:3300/0 conn(0x7f92ac0af4d0 msgr2=0x7f92ac0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 --2- 172.21.15.114:0/3096707938 >> v2:172.21.15.172:3300/0 conn(0x7f92ac0af4d0 0x7f92ac0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 >> v1:172.21.15.172:6789/0 conn(0x7f92ac15e080 legacy=0x7f92ac06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0ae9f0 con 0x7f92ac06c710 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f929c001670 con 0x7f92ac06c710 2023-11-24T06:25:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 >> v1:172.21.15.114:6789/0 conn(0x7f92ac06c710 legacy=0x7f92ac06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.182+0000 7f92b092e700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f9298003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9298004850 0x7f9298004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9298005210 0x7f929800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0af0b0 con 0x7f9298003a00 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929800c0b0 con 0x7f9298005210 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929800c270 con 0x7f9298004850 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f9298003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b112f700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9298004850 0x7f9298004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b2131700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9298005210 0x7f929800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 -- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9298004850 msgr2=0x7f9298004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9298004850 0x7f9298004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 -- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9298005210 msgr2=0x7f929800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9298005210 0x7f929800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 -- 172.21.15.114:0/3096707938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980035b0 con 0x7f9298003a00 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b112f700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9298004850 0x7f9298004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b2131700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9298005210 0x7f929800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b2131700 1 -- 172.21.15.114:0/3096707938 reap_dead start 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b1930700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f9298003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f92a80017c0 tx=0x7f9298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5474520eef9ec290 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80052d0 con 0x7f9298003a00 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a8005430 con 0x7f9298003a00 2023-11-24T06:25:45.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.183+0000 7f92b092e700 1 -- 172.21.15.114:0/3096707938 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80038c0 con 0x7f9298003a00 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 -- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 msgr2=0x7f9298003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f9298003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f92a80017c0 tx=0x7f9298005880 comp rx=0 tx=0).stop 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 -- 172.21.15.114:0/3096707938 shutdown_connections 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 --2- 172.21.15.114:0/3096707938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f9298003e20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 -- 172.21.15.114:0/3096707938 >> 172.21.15.114:0/3096707938 conn(0x7f92ac071b90 msgr2=0x7f92ac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 -- 172.21.15.114:0/3096707938 shutdown_connections 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.184+0000 7f92b3b94700 1 -- 172.21.15.114:0/3096707938 wait complete. 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.185+0000 7f92b3b94700 1 Processor -- start 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.185+0000 7f92b3b94700 1 -- start start 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92ac146570 0x7f92ac160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92ac160b70 0x7f92ac0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac161020 con 0x7f9298003a00 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac145840 con 0x7f92ac160b70 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1460e0 con 0x7f92ac146570 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40622/0 (socket says 172.21.15.114:40622) 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 -- 172.21.15.114:0/1937732796 learned_addr learned my addr 172.21.15.114:0/1937732796 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b112f700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92ac146570 0x7f92ac160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b2131700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92ac160b70 0x7f92ac0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 -- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92ac146570 msgr2=0x7f92ac160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92ac146570 0x7f92ac160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 -- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92ac160b70 msgr2=0x7f92ac0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92ac160b70 0x7f92ac0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 -- 172.21.15.114:0/1937732796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0ce310 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b1930700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005040 tx=0x7f92a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8001290 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a801d480 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac145a90 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a801d810 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.186+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac0cdb60 con 0x7f9298003a00 2023-11-24T06:25:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.187+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac0753f0 con 0x7f9298003a00 2023-11-24T06:25:45.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.188+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f92a8019040 con 0x7f9298003a00 2023-11-24T06:25:45.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.188+0000 7f92a27fc700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 0x7f929006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.188+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92a808a090 con 0x7f9298003a00 2023-11-24T06:25:45.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.188+0000 7f92b112f700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 0x7f929006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.189+0000 7f92b112f700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 0x7f929006d540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0733d0 tx=0x7f929c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.193+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92a80553a0 con 0x7f9298003a00 2023-11-24T06:25:45.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.425+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92ac0b0fb0 con 0x7f9298003a00 2023-11-24T06:25:45.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.426+0000 7f92a27fc700 1 -- 172.21.15.114:0/1937732796 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f92a802de90 con 0x7f9298003a00 2023-11-24T06:25:45.429 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:45.429 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4236067516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4236067516},{"type":"v1","addr":"172.21.15.172:6835","nonce":4236067516}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4858,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/958017593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":958017593},{"type":"v1","addr":"172.21.15.172:6834","nonce":958017593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5124,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3870420636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3870420636},{"type":"v1","addr":"172.21.15.114:6836","nonce":3870420636}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/3337124424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3337124424},{"type":"v1","addr":"172.21.15.114:6837","nonce":3337124424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-11-24T06:25:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 msgr2=0x7f929006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 0x7f929006d540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0733d0 tx=0x7f929c007040 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 msgr2=0x7f92ac143060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005040 tx=0x7f92a8001120 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 shutdown_connections 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f929006b080 0x7f929006d540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92ac160b70 0x7f92ac0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9298003a00 0x7f92ac143060 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 --2- 172.21.15.114:0/1937732796 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92ac146570 0x7f92ac160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 >> 172.21.15.114:0/1937732796 conn(0x7f92ac071b90 msgr2=0x7f92ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 shutdown_connections 2023-11-24T06:25:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.428+0000 7f92b3b94700 1 -- 172.21.15.114:0/1937732796 wait complete. 2023-11-24T06:25:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 20 2023-11-24T06:25:45.443 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:25:45.443 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:25:45.443 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:25:45.443 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:45.443 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:25:45.444 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:25:45.444 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:25:45.444 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:45.444 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:25:45.444 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:25:45.444 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:25:45.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:45.445 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:25:45.445 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:25:45.445 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:25:45.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:25:45.454 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:25:45.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:25:45.455 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:25:45.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:25:45.456 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:25:45.456 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:25:45.457 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:25:45.457 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:25:45.496 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:25:45.497 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:45.498 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:25:45.498 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:45.499 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:25:45.499 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:45.500 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:25:45.500 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:45.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:25:45.575 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:25:45.575 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2023-11-24T06:25:45.589 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-11-24T06:25:45.589 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.625+0000 7ff94e275700 1 Processor -- start 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.625+0000 7ff94e275700 1 -- start start 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94e275700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff94806c710 0x7ff9480b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94e275700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9480af4d0 0x7ff9480afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94e275700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9480ae430 con 0x7ff94815e080 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94e275700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9480af0b0 con 0x7ff94806c710 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94e275700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b05a0 con 0x7ff9480af4d0 2023-11-24T06:25:45.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff94806c710 0x7ff9480b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff947fff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff94815e080 0x7ff94806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:49956/0 (socket says 172.21.15.114:49956) 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff94c812700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9480af4d0 0x7ff9480afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff94806c710 0x7ff9480b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:36312/0 (socket says 172.21.15.114:36312) 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff947fff700 1 -- 172.21.15.114:0/1256369455 learned_addr learned my addr 172.21.15.114:0/1256369455 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 -- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9480af4d0 msgr2=0x7ff9480afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 --2- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9480af4d0 0x7ff9480afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 -- 172.21.15.114:0/1256369455 >> v1:172.21.15.172:6789/0 conn(0x7ff94815e080 legacy=0x7ff94806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff9477fe700 1 -- 172.21.15.114:0/1256369455 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480ae710 con 0x7ff94806c710 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.626+0000 7ff946ffd700 1 -- 172.21.15.114:0/1256369455 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960334017 0 0) 0x7ff9480ae430 con 0x7ff94815e080 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff9477fe700 1 --2- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94806c710 0x7ff9480b5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff938009840 tx=0x7ff938009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c56405753f1cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff946ffd700 1 -- 172.21.15.114:0/1256369455 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938011830 con 0x7ff94806c710 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff946ffd700 1 -- 172.21.15.114:0/1256369455 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938011990 con 0x7ff94806c710 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff946ffd700 1 -- 172.21.15.114:0/1256369455 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938011d20 con 0x7ff94806c710 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 -- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94806c710 msgr2=0x7ff9480b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 --2- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94806c710 0x7ff9480b5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff938009840 tx=0x7ff938009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 -- 172.21.15.114:0/1256369455 shutdown_connections 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 --2- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9480af4d0 0x7ff9480afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 --2- 172.21.15.114:0/1256369455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94806c710 0x7ff9480b5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 -- 172.21.15.114:0/1256369455 >> 172.21.15.114:0/1256369455 conn(0x7ff948071b90 msgr2=0x7ff948071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 -- 172.21.15.114:0/1256369455 shutdown_connections 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.627+0000 7ff94e275700 1 -- 172.21.15.114:0/1256369455 wait complete. 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.628+0000 7ff94e275700 1 Processor -- start 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 -- start start 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94806c710 0x7ff948151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 0x7ff94815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94815ed70 con 0x7ff94815e080 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff948151c60 con 0x7ff9480af4d0 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.629+0000 7ff94e275700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94815e460 con 0x7ff94806c710 2023-11-24T06:25:45.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94c812700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 0x7ff94815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94c812700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40662/0 (socket says 172.21.15.114:40662) 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff947fff700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94806c710 0x7ff948151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94c812700 1 -- 172.21.15.114:0/708804160 learned_addr learned my addr 172.21.15.114:0/708804160 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 -- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94806c710 msgr2=0x7ff948151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94806c710 0x7ff948151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 -- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 msgr2=0x7ff94815f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 -- 172.21.15.114:0/708804160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380094a0 con 0x7ff9480af4d0 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94c812700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff9477fe700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 0x7ff94815f190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff938016000 tx=0x7ff938004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9380036a0 con 0x7ff9480af4d0 2023-11-24T06:25:45.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93801f070 con 0x7ff9480af4d0 2023-11-24T06:25:45.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938003a20 con 0x7ff9480af4d0 2023-11-24T06:25:45.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9481f7a80 con 0x7ff9480af4d0 2023-11-24T06:25:45.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.630+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94815e990 con 0x7ff9480af4d0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.631+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9480753f0 con 0x7ff9480af4d0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.632+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff938003b80 con 0x7ff9480af4d0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.632+0000 7ff944ff9700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 0x7ff92806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.632+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff938021070 con 0x7ff9480af4d0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.632+0000 7ff947fff700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 0x7ff92806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.632+0000 7ff947fff700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 0x7ff92806d540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff930000f60 tx=0x7ff930007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:45.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.637+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff938055e00 con 0x7ff9480af4d0 2023-11-24T06:25:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:45.844+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff94815eeb0 con 0x7ff9480af4d0 2023-11-24T06:25:46.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:46.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:46.506 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.504+0000 7ff944ff9700 1 -- 172.21.15.114:0/708804160 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff94815eeb0 con 0x7ff9480af4d0 2023-11-24T06:25:46.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 msgr2=0x7ff92806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 0x7ff92806d540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff930000f60 tx=0x7ff930007e70 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 msgr2=0x7ff94815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 0x7ff94815f190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff938016000 tx=0x7ff938004750 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 shutdown_connections 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92806b080 0x7ff92806d540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94815e080 0x7ff94815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94806c710 0x7ff948151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 --2- 172.21.15.114:0/708804160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9480af4d0 0x7ff94815f190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 >> 172.21.15.114:0/708804160 conn(0x7ff948071b90 msgr2=0x7ff9480b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 shutdown_connections 2023-11-24T06:25:46.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.507+0000 7ff94e275700 1 -- 172.21.15.114:0/708804160 wait complete. 2023-11-24T06:25:46.511 INFO:teuthology.orchestra.run.smithi114.stderr: removed all blocklist entries 2023-11-24T06:25:46.524 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:25:46.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.687+0000 7fbf55f06700 1 Processor -- start 2023-11-24T06:25:46.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.687+0000 7fbf55f06700 1 -- start start 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf55f06700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf5015e080 0x7fbf500b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf55f06700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf5006c710 0x7fbf500afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf55f06700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf500ae430 con 0x7fbf500af4d0 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf55f06700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500af0b0 con 0x7fbf5006c710 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf55f06700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500b05a0 con 0x7fbf5015e080 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf5006c710 0x7fbf500afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf46ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf500af4d0 0x7fbf5006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56376/0 (socket says 172.21.15.114:56376) 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf5006c710 0x7fbf500afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40670/0 (socket says 172.21.15.114:40670) 2023-11-24T06:25:46.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf46ffd700 1 -- 172.21.15.114:0/1130664791 learned_addr learned my addr 172.21.15.114:0/1130664791 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4f7fe700 1 --2- 172.21.15.114:0/1130664791 >> v2:172.21.15.172:3300/0 conn(0x7fbf5015e080 0x7fbf500b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 -- 172.21.15.114:0/1130664791 >> v2:172.21.15.172:3300/0 conn(0x7fbf5015e080 msgr2=0x7fbf500b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 --2- 172.21.15.114:0/1130664791 >> v2:172.21.15.172:3300/0 conn(0x7fbf5015e080 0x7fbf500b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 -- 172.21.15.114:0/1130664791 >> v1:172.21.15.114:6789/0 conn(0x7fbf500af4d0 legacy=0x7fbf5006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 -- 172.21.15.114:0/1130664791 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500ae710 con 0x7fbf5006c710 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4effd700 1 -- 172.21.15.114:0/1130664791 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178662552 0 0) 0x7fbf500ae430 con 0x7fbf500af4d0 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.688+0000 7fbf4ffff700 1 --2- 172.21.15.114:0/1130664791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf500afb90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf50072a90 tx=0x7fbf4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e010381f1185312 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf4effd700 1 -- 172.21.15.114:0/1130664791 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800e070 con 0x7fbf5006c710 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf4effd700 1 -- 172.21.15.114:0/1130664791 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4800bb90 con 0x7fbf5006c710 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf4effd700 1 -- 172.21.15.114:0/1130664791 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48012430 con 0x7fbf5006c710 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 -- 172.21.15.114:0/1130664791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 msgr2=0x7fbf500afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1130664791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf500afb90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf50072a90 tx=0x7fbf4800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 -- 172.21.15.114:0/1130664791 shutdown_connections 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1130664791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf500afb90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1130664791 >> v2:172.21.15.172:3300/0 conn(0x7fbf5015e080 0x7fbf500b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 -- 172.21.15.114:0/1130664791 >> 172.21.15.114:0/1130664791 conn(0x7fbf50071b90 msgr2=0x7fbf50071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 -- 172.21.15.114:0/1130664791 shutdown_connections 2023-11-24T06:25:46.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.689+0000 7fbf55f06700 1 -- 172.21.15.114:0/1130664791 wait complete. 2023-11-24T06:25:46.696 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.690+0000 7fbf55f06700 1 Processor -- start 2023-11-24T06:25:46.696 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 -- start start 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 0x7fbf5015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf5015e080 0x7fbf5014a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5015faf0 con 0x7fbf5006c710 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5015eee0 con 0x7fbf500af4d0 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf55f06700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5015f1c0 con 0x7fbf5015e080 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf4f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf4f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40672/0 (socket says 172.21.15.114:40672) 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf4f7fe700 1 -- 172.21.15.114:0/1879618828 learned_addr learned my addr 172.21.15.114:0/1879618828 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.691+0000 7fbf46ffd700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 0x7fbf5015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4ffff700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf5015e080 0x7fbf5014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 -- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf5015e080 msgr2=0x7fbf5014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf5015e080 0x7fbf5014a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 -- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 msgr2=0x7fbf5015e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 0x7fbf5015e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 -- 172.21.15.114:0/1879618828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009a70 con 0x7fbf5006c710 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf46ffd700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 0x7fbf5015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4f7fe700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d800 tx=0x7fbf4000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400125f0 con 0x7fbf5006c710 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf40012750 con 0x7fbf5006c710 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5015f4a0 con 0x7fbf5006c710 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40012b00 con 0x7fbf5006c710 2023-11-24T06:25:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.692+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5014b6d0 con 0x7fbf5006c710 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.693+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5015fdd0 con 0x7fbf5006c710 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.693+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf40012c60 con 0x7fbf5006c710 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.694+0000 7fbf4cff9700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 0x7fbf3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.694+0000 7fbf46ffd700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 0x7fbf3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.694+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf40089970 con 0x7fbf5006c710 2023-11-24T06:25:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.694+0000 7fbf46ffd700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 0x7fbf3006d540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbf5014f860 tx=0x7fbf38007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.698+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf40054af0 con 0x7fbf5006c710 2023-11-24T06:25:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.967+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbf500753f0 con 0x7fbf5006c710 2023-11-24T06:25:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.967+0000 7fbf4cff9700 1 -- 172.21.15.114:0/1879618828 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbf40058110 con 0x7fbf5006c710 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "auth_dump": [ 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.a", 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBloXHaIBAA40BWWVcYcVbX+kM3wd4srw==", 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:46.970 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.b", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlRJM0LRAAToQLwjepMjKSY6cakPeFJw==", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.c", 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBllY8dKBAAtxCS400KLv4rt6WIzc3a2g==", 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.972 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.d", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlwqqDNBAARSrqgGNEALaFrG0ZFStYgg==", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.0", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBZQWBlQ7/gFxAA603E5V+l/7dkugKMUNQV3A==", 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.1", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBcQWBlnHtEFxAAr9h0RnKScdwcnLc2O9Pp4A==", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.2", 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBfQWBlmYVaGBAARyCei1YbVxxzy00k0W/Z6w==", 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.3", 2023-11-24T06:25:46.975 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBjQWBlVqdEABAAyD3s0xW8jXecRwdgtq8AwQ==", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.4", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBpQWBlyLk4BxAAauMGnn0akTG5zKYKcy9Vmw==", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.5", 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBsQWBlQjG4BhAAuv1Ja24nPcZ+SmnV8yuHgg==", 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.6", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBvQWBl4tb6BRAAcoydr7CybV8Ckv++5LFS+w==", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.7", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQByQWBlIUmuBRAAO0eSyPnvmpUft7iB4vQ6dg==", 2023-11-24T06:25:46.978 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.0", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g==", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow r", 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:25:46.979 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.1", 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlSFhWBhAA13v9kza/+snKVCmwnfK4eA==", 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.980 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.2", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlKtb6AhAAoaQhrX2DZgvNW0rcCB1iVg==", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.3", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlwshBDBAAkMjk+usVBY7TYehxOhJdDw==", 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.981 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.admin", 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlz1e2AhAAHaN4NjkwUY+J8W2nidj3Gg==", 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.982 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.x", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlT3B0HRAANmHJk8Er7FHsAeQJuAnUKg==", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:25:46.983 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.y", 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlcY5dGBAAZ7ISSGzT/OCSkRVGTBwkvA==", 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2023-11-24T06:25:46.984 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 msgr2=0x7fbf3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 0x7fbf3006d540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbf5014f860 tx=0x7fbf38007ca0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 msgr2=0x7fbf50150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d800 tx=0x7fbf4000dbc0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 shutdown_connections 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf3006b080 0x7fbf3006d540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf5015e080 0x7fbf5014a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5006c710 0x7fbf50150280 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 --2- 172.21.15.114:0/1879618828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500af4d0 0x7fbf5015e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 >> 172.21.15.114:0/1879618828 conn(0x7fbf50071b90 msgr2=0x7fbf50152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 shutdown_connections 2023-11-24T06:25:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:46.969+0000 7fbf55f06700 1 -- 172.21.15.114:0/1879618828 wait complete. 2023-11-24T06:25:46.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:47.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.105+0000 7f13bf28d700 1 Processor -- start 2023-11-24T06:25:47.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.105+0000 7f13bf28d700 1 -- start start 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bf28d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b815e080 0x7f13b80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bf28d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f13b80af4d0 0x7f13b80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bf28d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b80ae430 con 0x7f13b806c000 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bf28d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b80af0b0 con 0x7f13b80af4d0 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bf28d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80b05a0 con 0x7f13b815e080 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f13b80af4d0 0x7f13b80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f13b80af4d0 0x7f13b80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40680/0 (socket says 172.21.15.114:40680) 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 -- 172.21.15.114:0/4132993159 learned_addr learned my addr 172.21.15.114:0/4132993159 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:47.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bd029700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b815e080 0x7f13b80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 -- 172.21.15.114:0/4132993159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b815e080 msgr2=0x7f13b80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b815e080 0x7f13b80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 -- 172.21.15.114:0/4132993159 >> v1:172.21.15.172:6789/0 conn(0x7f13b806c000 legacy=0x7f13b8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.106+0000 7f13bc828700 1 -- 172.21.15.114:0/4132993159 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b80ae710 con 0x7f13b80af4d0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bc828700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b80af4d0 0x7f13b80afb90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009920 tx=0x7f13a8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42b045b37ad2d03 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13affff700 1 -- 172.21.15.114:0/4132993159 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a800ba20 con 0x7f13b80af4d0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13affff700 1 -- 172.21.15.114:0/4132993159 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a800bb80 con 0x7f13b80af4d0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13affff700 1 -- 172.21.15.114:0/4132993159 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8009e70 con 0x7f13b80af4d0 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 -- 172.21.15.114:0/4132993159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b80af4d0 msgr2=0x7f13b80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b80af4d0 0x7f13b80afb90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009920 tx=0x7f13a8009c30 comp rx=0 tx=0).stop 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 -- 172.21.15.114:0/4132993159 shutdown_connections 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b815e080 0x7f13b80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 --2- 172.21.15.114:0/4132993159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b80af4d0 0x7f13b80afb90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 -- 172.21.15.114:0/4132993159 >> 172.21.15.114:0/4132993159 conn(0x7f13b8071b90 msgr2=0x7f13b8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 -- 172.21.15.114:0/4132993159 shutdown_connections 2023-11-24T06:25:47.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.107+0000 7f13bf28d700 1 -- 172.21.15.114:0/4132993159 wait complete. 2023-11-24T06:25:47.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.108+0000 7f13bf28d700 1 Processor -- start 2023-11-24T06:25:47.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 -- start start 2023-11-24T06:25:47.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 0x7f13b81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b80af4d0 0x7f13b8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b81588c0 con 0x7f13b815e080 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8158060 con 0x7f13b806c000 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bf28d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b8158300 con 0x7f13b80af4d0 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bd82a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bd82a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40690/0 (socket says 172.21.15.114:40690) 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bd82a700 1 -- 172.21.15.114:0/1006738891 learned_addr learned my addr 172.21.15.114:0/1006738891 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bd029700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 0x7f13b81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.109+0000 7f13bc828700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b80af4d0 0x7f13b8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 -- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b80af4d0 msgr2=0x7f13b8157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b80af4d0 0x7f13b8157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 -- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 msgr2=0x7f13b81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 0x7f13b81566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 -- 172.21.15.114:0/1006738891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8009550 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd029700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 0x7f13b81566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bd82a700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f13b000cc50 tx=0x7f13b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000ad80 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000ce50 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b81585e0 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003c40 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.110+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b814a740 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.111+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b80753f0 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.111+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f13b00050d0 con 0x7f13b815e080 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.112+0000 7f13adffb700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 0x7f139c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.112+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f13b0089630 con 0x7f13b815e080 2023-11-24T06:25:47.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.112+0000 7f13bd029700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 0x7f139c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.112+0000 7f13bd029700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 0x7f139c06d540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f13b400ba70 tx=0x7f13b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.116+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13b0054840 con 0x7f13b815e080 2023-11-24T06:25:47.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.307+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13b8154120 con 0x7f13b815e080 2023-11-24T06:25:47.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.308+0000 7f13adffb700 1 -- 172.21.15.114:0/1006738891 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12893 (secure 0 0 0) 0x7f13b8154120 con 0x7f13b815e080 2023-11-24T06:25:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:47.310 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":41,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:46.508316+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 msgr2=0x7f139c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 0x7f139c06d540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f13b400ba70 tx=0x7f13b4008040 comp rx=0 tx=0).stop 2023-11-24T06:25:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 msgr2=0x7f13b8158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f13b000cc50 tx=0x7f13b000a340 comp rx=0 tx=0).stop 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 shutdown_connections 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f139c06b080 0x7f139c06d540 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13b815e080 0x7f13b8158fe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13b806c000 0x7f13b81566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 --2- 172.21.15.114:0/1006738891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13b80af4d0 0x7f13b8157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 >> 172.21.15.114:0/1006738891 conn(0x7f13b8071b90 msgr2=0x7f13b80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 shutdown_connections 2023-11-24T06:25:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.310+0000 7f13bf28d700 1 -- 172.21.15.114:0/1006738891 wait complete. 2023-11-24T06:25:47.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:25:47.325 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:25:47.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.448+0000 7f865b8f1700 1 Processor -- start 2023-11-24T06:25:47.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865b8f1700 1 -- start start 2023-11-24T06:25:47.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865b8f1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8654161d20 0x7f8654162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.452 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865b8f1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8654162640 con 0x7f86540995e0 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865b8f1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540ce2e0 con 0x7f865406eab0 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865b8f1700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865406f2d0 con 0x7f8654161d20 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.449+0000 7f865968d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8654161d20 0x7f8654162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f8658e8c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f86540995e0 0x7f865406e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56402/0 (socket says 172.21.15.114:56402) 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f8658e8c700 1 -- 172.21.15.114:0/1433577826 learned_addr learned my addr 172.21.15.114:0/1433577826 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f865968d700 1 -- 172.21.15.114:0/1433577826 >> v1:172.21.15.172:6789/0 conn(0x7f865406eab0 legacy=0x7f86540cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f865968d700 1 -- 172.21.15.114:0/1433577826 >> v1:172.21.15.114:6789/0 conn(0x7f86540995e0 legacy=0x7f865406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f865968d700 1 -- 172.21.15.114:0/1433577826 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540ce5a0 con 0x7f8654161d20 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f864bfff700 1 -- 172.21.15.114:0/1433577826 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313870378 0 0) 0x7f8654162640 con 0x7f86540995e0 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f865968d700 1 --2- 172.21.15.114:0/1433577826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8654161d20 0x7f8654162100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8644009840 tx=0x7f8644009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8821fde0bec1a8d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f864bfff700 1 -- 172.21.15.114:0/1433577826 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644011830 con 0x7f8654161d20 2023-11-24T06:25:47.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f864bfff700 1 -- 172.21.15.114:0/1433577826 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644011990 con 0x7f8654161d20 2023-11-24T06:25:47.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.450+0000 7f864bfff700 1 -- 172.21.15.114:0/1433577826 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644011d20 con 0x7f8654161d20 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 -- 172.21.15.114:0/1433577826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8654161d20 msgr2=0x7f8654162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 --2- 172.21.15.114:0/1433577826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8654161d20 0x7f8654162100 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8644009840 tx=0x7f8644009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 -- 172.21.15.114:0/1433577826 shutdown_connections 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 --2- 172.21.15.114:0/1433577826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8654161d20 0x7f8654162100 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 -- 172.21.15.114:0/1433577826 >> 172.21.15.114:0/1433577826 conn(0x7f86540726e0 msgr2=0x7f8654072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 -- 172.21.15.114:0/1433577826 shutdown_connections 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.451+0000 7f865b8f1700 1 -- 172.21.15.114:0/1433577826 wait complete. 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.452+0000 7f865b8f1700 1 Processor -- start 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.452+0000 7f865b8f1700 1 -- start start 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865406eab0 0x7f865414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8654161d20 0x7f86540b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865415fd10 con 0x7f86540995e0 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865415f090 con 0x7f865406eab0 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865415f370 con 0x7f8654161d20 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40700/0 (socket says 172.21.15.114:40700) 2023-11-24T06:25:47.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 -- 172.21.15.114:0/3329312749 learned_addr learned my addr 172.21.15.114:0/3329312749 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865968d700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865406eab0 0x7f865414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8659e8e700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8654161d20 0x7f86540b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 -- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8654161d20 msgr2=0x7f86540b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8654161d20 0x7f86540b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 -- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865406eab0 msgr2=0x7f865414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865406eab0 0x7f865414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 -- 172.21.15.114:0/3329312749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440094a0 con 0x7f86540995e0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8658e8c700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f865000c520 tx=0x7f865000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650004c80 con 0x7f86540995e0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8650004de0 con 0x7f86540995e0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865415f650 con 0x7f86540995e0 2023-11-24T06:25:47.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.453+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650007310 con 0x7f86540995e0 2023-11-24T06:25:47.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.454+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865415ecf0 con 0x7f86540995e0 2023-11-24T06:25:47.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.454+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654068fb0 con 0x7f86540995e0 2023-11-24T06:25:47.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.455+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8650007470 con 0x7f86540995e0 2023-11-24T06:25:47.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.455+0000 7f8649ffb700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 0x7f863806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.455+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8650089230 con 0x7f86540995e0 2023-11-24T06:25:47.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.455+0000 7f865968d700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 0x7f863806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.460 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.456+0000 7f865968d700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 0x7f863806d540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86440090b0 tx=0x7f864401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.460+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86500543a0 con 0x7f86540995e0 2023-11-24T06:25:47.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.652+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86540cc410 con 0x7f86540995e0 2023-11-24T06:25:47.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.653+0000 7f8649ffb700 1 -- 172.21.15.114:0/3329312749 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86500579c0 con 0x7f86540995e0 2023-11-24T06:25:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:25:47.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 msgr2=0x7f863806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 0x7f863806d540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86440090b0 tx=0x7f864401c040 comp rx=0 tx=0).stop 2023-11-24T06:25:47.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 msgr2=0x7f865415e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f865000c520 tx=0x7f865000c830 comp rx=0 tx=0).stop 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 shutdown_connections 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f863806b080 0x7f863806d540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8654161d20 0x7f86540b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f865406eab0 0x7f865414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 --2- 172.21.15.114:0/3329312749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86540995e0 0x7f865415e7b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 >> 172.21.15.114:0/3329312749 conn(0x7f86540726e0 msgr2=0x7f865406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 shutdown_connections 2023-11-24T06:25:47.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.655+0000 7f865b8f1700 1 -- 172.21.15.114:0/3329312749 wait complete. 2023-11-24T06:25:47.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:25:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.788+0000 7f61a8496700 1 Processor -- start 2023-11-24T06:25:47.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.788+0000 7f61a8496700 1 -- start start 2023-11-24T06:25:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a8496700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61a015e080 0x7f61a00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a8496700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00ae430 con 0x7f61a00af4d0 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a8496700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00ae710 con 0x7f61a006c710 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a8496700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00af0b0 con 0x7f61a015e080 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a5a31700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f61a00af4d0 0x7f61a006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56418/0 (socket says 172.21.15.114:56418) 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.789+0000 7f61a5a31700 1 -- 172.21.15.114:0/2540144970 learned_addr learned my addr 172.21.15.114:0/2540144970 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6232700 1 --2- 172.21.15.114:0/2540144970 >> v2:172.21.15.172:3300/0 conn(0x7f61a015e080 0x7f61a00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818230595 0 0) 0x7f61a00ae430 con 0x7f61a00af4d0 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f618c0035b0 con 0x7f61a00af4d0 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 762314597 0 0) 0x7f618c0035b0 con 0x7f61a00af4d0 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 >> v2:172.21.15.172:3300/0 conn(0x7f61a015e080 msgr2=0x7f61a00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 --2- 172.21.15.114:0/2540144970 >> v2:172.21.15.172:3300/0 conn(0x7f61a015e080 0x7f61a00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 >> v1:172.21.15.172:6789/0 conn(0x7f61a006c710 legacy=0x7f61a006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00ae9f0 con 0x7f61a00af4d0 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f619c001670 con 0x7f61a00af4d0 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 >> v1:172.21.15.114:6789/0 conn(0x7f61a00af4d0 legacy=0x7f61a006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c003a00 0x7f618c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c004850 0x7f618c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 0x7f618c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00af0b0 con 0x7f618c005210 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c00c0b0 con 0x7f618c003a00 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c00c270 con 0x7f618c004850 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 0x7f618c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6232700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c003a00 0x7f618c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a5a31700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c004850 0x7f618c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 -- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c004850 msgr2=0x7f618c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c004850 0x7f618c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 -- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c003a00 msgr2=0x7f618c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c003a00 0x7f618c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.790+0000 7f61a6a33700 1 -- 172.21.15.114:0/2540144970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0035b0 con 0x7f618c005210 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a6a33700 1 -- 172.21.15.114:0/2540144970 reap_dead start 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 0x7f618c00ba10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f619800c160 tx=0x7f618c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe9ed86e62d7b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198001db0 con 0x7f618c005210 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800ccc0 con 0x7f618c005210 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a5230700 1 -- 172.21.15.114:0/2540144970 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198003c40 con 0x7f618c005210 2023-11-24T06:25:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 -- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 msgr2=0x7f618c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 0x7f618c00ba10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f619800c160 tx=0x7f618c005880 comp rx=0 tx=0).stop 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 -- 172.21.15.114:0/2540144970 shutdown_connections 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 --2- 172.21.15.114:0/2540144970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f618c005210 0x7f618c00ba10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 -- 172.21.15.114:0/2540144970 >> 172.21.15.114:0/2540144970 conn(0x7f61a0071b90 msgr2=0x7f61a0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 -- 172.21.15.114:0/2540144970 shutdown_connections 2023-11-24T06:25:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.791+0000 7f61a8496700 1 -- 172.21.15.114:0/2540144970 wait complete. 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 Processor -- start 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 -- start start 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c005210 0x7f61a014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 0x7f61a015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00c9340 con 0x7f61a00c8980 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a014bf50 con 0x7f61a015f810 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a8496700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a014c230 con 0x7f618c005210 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a5a31700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a5a31700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40722/0 (socket says 172.21.15.114:40722) 2023-11-24T06:25:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.793+0000 7f61a5a31700 1 -- 172.21.15.114:0/2983408534 learned_addr learned my addr 172.21.15.114:0/2983408534 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a6232700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c005210 0x7f61a014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 0x7f61a015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 -- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c005210 msgr2=0x7f61a014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c005210 0x7f61a014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 -- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 msgr2=0x7f61a015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 0x7f61a015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 -- 172.21.15.114:0/2983408534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0160560 con 0x7f61a00c8980 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a6a33700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 0x7f61a015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a5a31700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f619c000f60 tx=0x7f619c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00c5f0 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00c770 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a0160840 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cb20 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.794+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a0160e00 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.795+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a014c560 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.795+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f619c00cc80 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.796+0000 7f6196ffd700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 0x7f618406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.796+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f619c089380 con 0x7f61a00c8980 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.796+0000 7f61a6232700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 0x7f618406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.796+0000 7f61a6232700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 0x7f618406d540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6190009b20 tx=0x7f6190008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.800+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f619c054590 con 0x7f61a00c8980 2023-11-24T06:25:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.992+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61a00753f0 con 0x7f61a00c8980 2023-11-24T06:25:47.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.992+0000 7f6196ffd700 1 -- 172.21.15.114:0/2983408534 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f619c057bb0 con 0x7f61a00c8980 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:25:47.995 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 msgr2=0x7f618406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 0x7f618406d540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6190009b20 tx=0x7f6190008040 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 msgr2=0x7f61a00c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f619c000f60 tx=0x7f619c00eb80 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 shutdown_connections 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f618406b080 0x7f618406d540 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a015f810 0x7f61a015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f618c005210 0x7f61a014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 --2- 172.21.15.114:0/2983408534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61a00c8980 0x7f61a00c8e00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 >> 172.21.15.114:0/2983408534 conn(0x7f61a0071b90 msgr2=0x7f61a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 shutdown_connections 2023-11-24T06:25:47.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:47.994+0000 7f61a8496700 1 -- 172.21.15.114:0/2983408534 wait complete. 2023-11-24T06:25:48.008 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:25:48.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:25:48.133 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.131+0000 7f568b4de700 1 Processor -- start 2023-11-24T06:25:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.131+0000 7f568b4de700 1 -- start start 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568b4de700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56840b47e0 0x7f56840b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568b4de700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 0x7f568406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568b4de700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5684158be0 con 0x7f5684158800 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568b4de700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56840b51d0 con 0x7f56840b47e0 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568b4de700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568406f9d0 con 0x7f568406b210 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56840b47e0 0x7f56840b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568927a700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5684158800 0x7f568406e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56436/0 (socket says 172.21.15.114:56436) 2023-11-24T06:25:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f568927a700 1 -- 172.21.15.114:0/3112163463 learned_addr learned my addr 172.21.15.114:0/3112163463 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f56840b47e0 0x7f56840b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40730/0 (socket says 172.21.15.114:40730) 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5689a7b700 1 --2- 172.21.15.114:0/3112163463 >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 0x7f568406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 -- 172.21.15.114:0/3112163463 >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 msgr2=0x7f568406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 --2- 172.21.15.114:0/3112163463 >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 0x7f568406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 -- 172.21.15.114:0/3112163463 >> v1:172.21.15.114:6789/0 conn(0x7f5684158800 legacy=0x7f568406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f567bfff700 1 -- 172.21.15.114:0/3112163463 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934629564 0 0) 0x7f5684158be0 con 0x7f5684158800 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5688a79700 1 -- 172.21.15.114:0/3112163463 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568406edc0 con 0x7f56840b47e0 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.132+0000 7f5689a7b700 1 --2- 172.21.15.114:0/3112163463 >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 0x7f568406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f5688a79700 1 --2- 172.21.15.114:0/3112163463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f56840b4c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f56840d1810 tx=0x7f5680009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25bee17b96c3125 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f567bfff700 1 -- 172.21.15.114:0/3112163463 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680011830 con 0x7f56840b47e0 2023-11-24T06:25:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f567bfff700 1 -- 172.21.15.114:0/3112163463 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680011990 con 0x7f56840b47e0 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f567bfff700 1 -- 172.21.15.114:0/3112163463 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680011d20 con 0x7f56840b47e0 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 -- 172.21.15.114:0/3112163463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 msgr2=0x7f56840b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 --2- 172.21.15.114:0/3112163463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f56840b4c90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f56840d1810 tx=0x7f5680009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 -- 172.21.15.114:0/3112163463 shutdown_connections 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 --2- 172.21.15.114:0/3112163463 >> v2:172.21.15.172:3300/0 conn(0x7f568406b210 0x7f568406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 --2- 172.21.15.114:0/3112163463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f56840b4c90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 -- 172.21.15.114:0/3112163463 >> 172.21.15.114:0/3112163463 conn(0x7f56840d0b40 msgr2=0x7f56840d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 -- 172.21.15.114:0/3112163463 shutdown_connections 2023-11-24T06:25:48.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.133+0000 7f568b4de700 1 -- 172.21.15.114:0/3112163463 wait complete. 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.134+0000 7f568b4de700 1 Processor -- start 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 -- start start 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f568406b210 0x7f568415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 0x7f5684073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568415a370 con 0x7f56840b47e0 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840a9670 con 0x7f5684158800 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f568b4de700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5684154040 con 0x7f568406b210 2023-11-24T06:25:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f5688a79700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f5688a79700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40736/0 (socket says 172.21.15.114:40736) 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.135+0000 7f5688a79700 1 -- 172.21.15.114:0/2493881534 learned_addr learned my addr 172.21.15.114:0/2493881534 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 -- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f568406b210 msgr2=0x7f568415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f568406b210 0x7f568415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5689a7b700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 0x7f5684073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 -- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 msgr2=0x7f5684073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 0x7f5684073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 -- 172.21.15.114:0/2493881534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800094a0 con 0x7f56840b47e0 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5689a7b700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 0x7f5684073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5688a79700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5680000c00 tx=0x7f5680004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56800038a0 con 0x7f56840b47e0 2023-11-24T06:25:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568001f070 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680003c60 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5684153d60 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.136+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5684159f90 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.137+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684067f60 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.138+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f568001a430 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.138+0000 7f5679ffb700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 0x7f566806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.138+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5680021070 con 0x7f56840b47e0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.138+0000 7f568927a700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 0x7f566806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.139+0000 7f568927a700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 0x7f566806d540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f568406bcb0 tx=0x7f56740079f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.142+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56800563d0 con 0x7f56840b47e0 2023-11-24T06:25:48.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.334+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f568415a4b0 con 0x7f56840b47e0 2023-11-24T06:25:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.504+0000 7f5679ffb700 1 -- 172.21.15.114:0/2493881534 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f568415a4b0 con 0x7f56840b47e0 2023-11-24T06:25:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 msgr2=0x7f566806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 0x7f566806d540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f568406bcb0 tx=0x7f56740079f0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 msgr2=0x7f5684158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5680000c00 tx=0x7f5680004750 comp rx=0 tx=0).stop 2023-11-24T06:25:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 shutdown_connections 2023-11-24T06:25:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f566806b080 0x7f566806d540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5684158800 0x7f5684073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f568406b210 0x7f568415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 --2- 172.21.15.114:0/2493881534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56840b47e0 0x7f5684158be0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 >> 172.21.15.114:0/2493881534 conn(0x7f56840d0b40 msgr2=0x7f5684157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 shutdown_connections 2023-11-24T06:25:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.508+0000 7f568b4de700 1 -- 172.21.15.114:0/2493881534 wait complete. 2023-11-24T06:25:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2023-11-24T06:25:48.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:25:48.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.651+0000 7f481f464700 1 Processor -- start 2023-11-24T06:25:48.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.651+0000 7f481f464700 1 -- start start 2023-11-24T06:25:48.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481f464700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f481815bf50 0x7f48180ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481f464700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48180c6c30 0x7f48180cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481f464700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180d0140 con 0x7f48180ce830 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481f464700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481815c330 con 0x7f481815bf50 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481f464700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180cf3c0 con 0x7f48180c6c30 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481d200700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f481815bf50 0x7f48180ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481d200700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f481815bf50 0x7f48180ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40742/0 (socket says 172.21.15.114:40742) 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.652+0000 7f481d200700 1 -- 172.21.15.114:0/1405203677 learned_addr learned my addr 172.21.15.114:0/1405203677 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:48.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481da01700 1 --2- 172.21.15.114:0/1405203677 >> v2:172.21.15.172:3300/0 conn(0x7f48180c6c30 0x7f48180cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481d200700 1 -- 172.21.15.114:0/1405203677 >> v2:172.21.15.172:3300/0 conn(0x7f48180c6c30 msgr2=0x7f48180cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481d200700 1 --2- 172.21.15.114:0/1405203677 >> v2:172.21.15.172:3300/0 conn(0x7f48180c6c30 0x7f48180cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481d200700 1 -- 172.21.15.114:0/1405203677 >> v1:172.21.15.172:6789/0 conn(0x7f48180ce830 legacy=0x7f48180cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481d200700 1 -- 172.21.15.114:0/1405203677 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180d03e0 con 0x7f481815bf50 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f481d200700 1 --2- 172.21.15.114:0/1405203677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f481815bf50 0x7f48180ce2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48180c8c00 tx=0x7f4814009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28641ebce8e9d20 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f480ffff700 1 -- 172.21.15.114:0/1405203677 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814004230 con 0x7f481815bf50 2023-11-24T06:25:48.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f480ffff700 1 -- 172.21.15.114:0/1405203677 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814004390 con 0x7f481815bf50 2023-11-24T06:25:48.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.653+0000 7f480ffff700 1 -- 172.21.15.114:0/1405203677 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140036a0 con 0x7f481815bf50 2023-11-24T06:25:48.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 -- 172.21.15.114:0/1405203677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f481815bf50 msgr2=0x7f48180ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 --2- 172.21.15.114:0/1405203677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f481815bf50 0x7f48180ce2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f48180c8c00 tx=0x7f4814009bd0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 -- 172.21.15.114:0/1405203677 shutdown_connections 2023-11-24T06:25:48.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 --2- 172.21.15.114:0/1405203677 >> v2:172.21.15.172:3300/0 conn(0x7f48180c6c30 0x7f48180cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 --2- 172.21.15.114:0/1405203677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f481815bf50 0x7f48180ce2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 -- 172.21.15.114:0/1405203677 >> 172.21.15.114:0/1405203677 conn(0x7f48180b3780 msgr2=0x7f48180b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 -- 172.21.15.114:0/1405203677 shutdown_connections 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.654+0000 7f481f464700 1 -- 172.21.15.114:0/1405203677 wait complete. 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.655+0000 7f481f464700 1 Processor -- start 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.655+0000 7f481f464700 1 -- start start 2023-11-24T06:25:48.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48180c6c30 0x7f4818157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f481815bf50 0x7f48181e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180c9d70 con 0x7f48180ce830 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180ca050 con 0x7f481815bf50 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180ca330 con 0x7f48180c6c30 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40750/0 (socket says 172.21.15.114:40750) 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 -- 172.21.15.114:0/3908405531 learned_addr learned my addr 172.21.15.114:0/3908405531 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481d200700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48180c6c30 0x7f4818157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481da01700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f481815bf50 0x7f48181e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 -- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48180c6c30 msgr2=0x7f4818157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48180c6c30 0x7f4818157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 -- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f481815bf50 msgr2=0x7f48181e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f481815bf50 0x7f48181e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 -- 172.21.15.114:0/3908405531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814009550 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481c9ff700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f480800c4d0 tx=0x7f480800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808011790 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48080118f0 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48180ca610 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.656+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808011ca0 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.657+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48181e6c10 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.657+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818068fb0 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.658+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f480801c460 con 0x7f48180ce830 2023-11-24T06:25:48.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.658+0000 7f480dffb700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 0x7f47fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:48.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.658+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4808089580 con 0x7f48180ce830 2023-11-24T06:25:48.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.658+0000 7f481d200700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 0x7f47fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:48.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.659+0000 7f481d200700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 0x7f47fc06d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4814009130 tx=0x7f4814008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:48.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.663+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4808054590 con 0x7f48180ce830 2023-11-24T06:25:48.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:48.854+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f48180b7ab0 con 0x7f48180ce830 2023-11-24T06:25:49.509 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.507+0000 7f480dffb700 1 -- 172.21.15.114:0/3908405531 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4808057bb0 con 0x7f48180ce830 2023-11-24T06:25:49.512 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 msgr2=0x7f47fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 0x7f47fc06d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4814009130 tx=0x7f4814008030 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 msgr2=0x7f48180cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f480800c4d0 tx=0x7f480800c890 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 shutdown_connections 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f47fc06b080 0x7f47fc06d540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f481815bf50 0x7f48181e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48180c6c30 0x7f4818157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 --2- 172.21.15.114:0/3908405531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48180ce830 0x7f48180cae10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 >> 172.21.15.114:0/3908405531 conn(0x7f48180b3780 msgr2=0x7f48180cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:49.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 shutdown_connections 2023-11-24T06:25:49.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.510+0000 7f481f464700 1 -- 172.21.15.114:0/3908405531 wait complete. 2023-11-24T06:25:49.514 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2023-11-24T06:25:49.540 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:25:49.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.666+0000 7fb693d39700 1 Processor -- start 2023-11-24T06:25:49.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.666+0000 7fb693d39700 1 -- start start 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb693d39700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb68c15e160 0x7fb68c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb693d39700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0b3c60 con 0x7fb68c0afc90 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb693d39700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0b58c0 con 0x7fb68c0746a0 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb693d39700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c0b4500 con 0x7fb68c15e160 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb691ad5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb68c15e160 0x7fb68c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb6912d4700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb68c0afc90 0x7fb68c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56464/0 (socket says 172.21.15.114:56464) 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb6912d4700 1 -- 172.21.15.114:0/2072215258 learned_addr learned my addr 172.21.15.114:0/2072215258 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb691ad5700 1 -- 172.21.15.114:0/2072215258 >> v1:172.21.15.172:6789/0 conn(0x7fb68c0746a0 legacy=0x7fb68c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb691ad5700 1 -- 172.21.15.114:0/2072215258 >> v1:172.21.15.114:6789/0 conn(0x7fb68c0afc90 legacy=0x7fb68c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb691ad5700 1 -- 172.21.15.114:0/2072215258 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0b0140 con 0x7fb68c15e160 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.667+0000 7fb691ad5700 1 --2- 172.21.15.114:0/2072215258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c15e160 0x7fb68c0b5da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb688009840 tx=0x7fb688009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9442db4711c657f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:49.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb690ad3700 1 -- 172.21.15.114:0/2072215258 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688004230 con 0x7fb68c15e160 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb690ad3700 1 -- 172.21.15.114:0/2072215258 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688004390 con 0x7fb68c15e160 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb690ad3700 1 -- 172.21.15.114:0/2072215258 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6880036a0 con 0x7fb68c15e160 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 -- 172.21.15.114:0/2072215258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c15e160 msgr2=0x7fb68c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 --2- 172.21.15.114:0/2072215258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c15e160 0x7fb68c0b5da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb688009840 tx=0x7fb688009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 -- 172.21.15.114:0/2072215258 shutdown_connections 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 --2- 172.21.15.114:0/2072215258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c15e160 0x7fb68c0b5da0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 -- 172.21.15.114:0/2072215258 >> 172.21.15.114:0/2072215258 conn(0x7fb68c0d0bf0 msgr2=0x7fb68c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 -- 172.21.15.114:0/2072215258 shutdown_connections 2023-11-24T06:25:49.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.668+0000 7fb693d39700 1 -- 172.21.15.114:0/2072215258 wait complete. 2023-11-24T06:25:49.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.669+0000 7fb693d39700 1 Processor -- start 2023-11-24T06:25:49.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.669+0000 7fb693d39700 1 -- start start 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb68c0afc90 0x7fb68c146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 0x7fb68c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0c63b0 con 0x7fb68c0746a0 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c15f0b0 con 0x7fb68c15e160 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb693d39700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0c6110 con 0x7fb68c0afc90 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb691ad5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb691ad5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40760/0 (socket says 172.21.15.114:40760) 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb691ad5700 1 -- 172.21.15.114:0/2458701223 learned_addr learned my addr 172.21.15.114:0/2458701223 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb691ad5700 1 -- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb68c0afc90 msgr2=0x7fb68c146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb6912d4700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb68c0afc90 0x7fb68c146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:49.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb6922d6700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 0x7fb68c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.670+0000 7fb691ad5700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb68c0afc90 0x7fb68c146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb691ad5700 1 -- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 msgr2=0x7fb68c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb691ad5700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 0x7fb68c0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb691ad5700 1 -- 172.21.15.114:0/2458701223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880094a0 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb6922d6700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 0x7fb68c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb691ad5700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb6880090b0 tx=0x7fb6880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688011830 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688011990 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688011d40 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c15eaf0 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.671+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c146c20 con 0x7fb68c0746a0 2023-11-24T06:25:49.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.672+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c0af4f0 con 0x7fb68c0746a0 2023-11-24T06:25:49.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.672+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb68801b440 con 0x7fb68c0746a0 2023-11-24T06:25:49.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.673+0000 7fb6827fc700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 0x7fb67006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:49.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.673+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb68808b530 con 0x7fb68c0746a0 2023-11-24T06:25:49.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.673+0000 7fb6912d4700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 0x7fb67006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:49.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.673+0000 7fb6912d4700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 0x7fb67006d540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb68c15d500 tx=0x7fb67c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:49.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.677+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb688056530 con 0x7fb68c0746a0 2023-11-24T06:25:49.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:49.912+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb68c0af4f0 con 0x7fb68c0746a0 2023-11-24T06:25:50.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.533+0000 7fb6827fc700 1 -- 172.21.15.114:0/2458701223 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb688059b50 con 0x7fb68c0746a0 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 msgr2=0x7fb67006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 0x7fb67006d540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb68c15d500 tx=0x7fb67c009040 comp rx=0 tx=0).stop 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 msgr2=0x7fb68c149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb6880090b0 tx=0x7fb6880047c0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 shutdown_connections 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb67006b080 0x7fb67006d540 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb68c15e160 0x7fb68c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0746a0 0x7fb68c149720 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 --2- 172.21.15.114:0/2458701223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb68c0afc90 0x7fb68c146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.557+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 >> 172.21.15.114:0/2458701223 conn(0x7fb68c0d0bf0 msgr2=0x7fb68c06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.558+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 shutdown_connections 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.558+0000 7fb693d39700 1 -- 172.21.15.114:0/2458701223 wait complete. 2023-11-24T06:25:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T06:25:50.576 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:50.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.695+0000 7f8604619700 1 Processor -- start 2023-11-24T06:25:50.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.695+0000 7f8604619700 1 -- start start 2023-11-24T06:25:50.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8604619700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc0afe70 0x7f85fc15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8604619700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f85fc1624d0 0x7f85fc162930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8604619700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc06dd80 con 0x7f85fc15bef0 2023-11-24T06:25:50.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8604619700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc15f880 con 0x7f85fc1624d0 2023-11-24T06:25:50.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8604619700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc15ed90 con 0x7f85fc0afe70 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86023b5700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f85fc15bef0 0x7f85fc0b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56490/0 (socket says 172.21.15.114:56490) 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86023b5700 1 -- 172.21.15.114:0/4183208481 learned_addr learned my addr 172.21.15.114:0/4183208481 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> v2:172.21.15.172:3300/0 conn(0x7f85fc1624d0 0x7f85fc162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f8601bb4700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc0afe70 0x7f85fc15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687651041 0 0) 0x7f85fc06dd80 con 0x7f85fc15bef0 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e80035b0 con 0x7f85fc15bef0 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160729953 0 0) 0x7f85e80035b0 con 0x7f85fc15bef0 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc0afe70 msgr2=0x7f85fc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc0afe70 0x7f85fc15e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 >> v2:172.21.15.172:3300/0 conn(0x7f85fc1624d0 msgr2=0x7f85fc162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:50.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 --2- 172.21.15.114:0/4183208481 >> v2:172.21.15.172:3300/0 conn(0x7f85fc1624d0 0x7f85fc162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc15c2d0 con 0x7f85fc15bef0 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f85f8001670 con 0x7f85fc15bef0 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.696+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 >> v1:172.21.15.114:6789/0 conn(0x7f85fc15bef0 legacy=0x7f85fc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85e8003a00 0x7f85e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85e80043f0 0x7f85e80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc15f880 con 0x7f85e8004fe0 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc15ed90 con 0x7f85e80043f0 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e800bf80 con 0x7f85e8003a00 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> v2:172.21.15.172:3300/0 conn(0x7f85fc1624d0 0x7f85fc162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8601bb4700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc0afe70 0x7f85fc15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86023b5700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85e8003a00 0x7f85e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8601bb4700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85e80043f0 0x7f85e80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 -- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85e8003a00 msgr2=0x7f85e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85e8003a00 0x7f85e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 -- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85e80043f0 msgr2=0x7f85e80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85e80043f0 0x7f85e80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 -- 172.21.15.114:0/4183208481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80035b0 con 0x7f85e8004fe0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f8602bb6700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85e800ba10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e350 tx=0x7f85e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952f9be1fc8a19e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400edb0 con 0x7f85e8004fe0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400f040 con 0x7f85e8004fe0 2023-11-24T06:25:50.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.697+0000 7f86013b3700 1 -- 172.21.15.114:0/4183208481 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003c40 con 0x7f85e8004fe0 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 -- 172.21.15.114:0/4183208481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 msgr2=0x7f85e800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 --2- 172.21.15.114:0/4183208481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85e800ba10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e350 tx=0x7f85e8005880 comp rx=0 tx=0).stop 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8602bb6700 1 -- 172.21.15.114:0/4183208481 reap_dead start 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 -- 172.21.15.114:0/4183208481 shutdown_connections 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 -- 172.21.15.114:0/4183208481 >> 172.21.15.114:0/4183208481 conn(0x7f85fc071ac0 msgr2=0x7f85fc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 -- 172.21.15.114:0/4183208481 shutdown_connections 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.698+0000 7f8604619700 1 -- 172.21.15.114:0/4183208481 wait complete. 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.699+0000 7f8604619700 1 Processor -- start 2023-11-24T06:25:50.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.699+0000 7f8604619700 1 -- start start 2023-11-24T06:25:50.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc15d500 0x7f85fc15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85fc0ce030 0x7f85fc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0cf390 con 0x7f85e8004fe0 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1580f0 con 0x7f85fc0ce030 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc15dec0 con 0x7f85fc15d500 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40794/0 (socket says 172.21.15.114:40794) 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 -- 172.21.15.114:0/2255197048 learned_addr learned my addr 172.21.15.114:0/2255197048 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8601bb4700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc15d500 0x7f85fc15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8602bb6700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85fc0ce030 0x7f85fc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 -- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc15d500 msgr2=0x7f85fc15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc15d500 0x7f85fc15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 -- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85fc0ce030 msgr2=0x7f85fc0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85fc0ce030 0x7f85fc0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 -- 172.21.15.114:0/2255197048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0cedd0 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f86023b5700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f85f80017a0 tx=0x7f85f80076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007ca0 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f800f8d0 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc0cf0b0 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.700+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f800fc80 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.701+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc1585e0 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.701+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc158770 con 0x7f85e8004fe0 2023-11-24T06:25:50.709 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.702+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f85f801d460 con 0x7f85e8004fe0 2023-11-24T06:25:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.702+0000 7f85f2ffd700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 0x7f85e006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.702+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85f8089a10 con 0x7f85e8004fe0 2023-11-24T06:25:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.702+0000 7f8601bb4700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 0x7f85e006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.703+0000 7f8601bb4700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 0x7f85e006d540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f85fc15e140 tx=0x7f85ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:50.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.707+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85f80548f0 con 0x7f85e8004fe0 2023-11-24T06:25:50.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.899+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85fc067f60 con 0x7f85e8004fe0 2023-11-24T06:25:50.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.900+0000 7f85f2ffd700 1 -- 172.21.15.114:0/2255197048 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f85f8057f10 con 0x7f85e8004fe0 2023-11-24T06:25:50.903 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:50.903 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":44,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:49.912907+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 msgr2=0x7f85e006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 0x7f85e006d540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f85fc15e140 tx=0x7f85ec008040 comp rx=0 tx=0).stop 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 msgr2=0x7f85fc157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f85f80017a0 tx=0x7f85f80076a0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 shutdown_connections 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f85e006b080 0x7f85e006d540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85fc0ce030 0x7f85fc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e8004fe0 0x7f85fc157610 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.905 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 --2- 172.21.15.114:0/2255197048 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85fc15d500 0x7f85fc15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:50.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 >> 172.21.15.114:0/2255197048 conn(0x7f85fc071ac0 msgr2=0x7f85fc06eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:50.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 shutdown_connections 2023-11-24T06:25:50.906 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:50.902+0000 7f8604619700 1 -- 172.21.15.114:0/2255197048 wait complete. 2023-11-24T06:25:50.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.035+0000 7f6cfa938700 1 Processor -- start 2023-11-24T06:25:51.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.035+0000 7f6cfa938700 1 -- start start 2023-11-24T06:25:51.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cfa938700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf415e080 0x7f6cf40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cfa938700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6cf406c710 0x7f6cf40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cfa938700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf40ae430 con 0x7f6cf40af4d0 2023-11-24T06:25:51.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cfa938700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf40af0b0 con 0x7f6cf406c710 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cfa938700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40b05a0 con 0x7f6cf415e080 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6cf406c710 0x7f6cf40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6cf406c710 0x7f6cf40afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40804/0 (socket says 172.21.15.114:40804) 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 -- 172.21.15.114:0/2620727506 learned_addr learned my addr 172.21.15.114:0/2620727506 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf3fff700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf415e080 0x7f6cf40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 -- 172.21.15.114:0/2620727506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf415e080 msgr2=0x7f6cf40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:51.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf415e080 0x7f6cf40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 -- 172.21.15.114:0/2620727506 >> v1:172.21.15.172:6789/0 conn(0x7f6cf40af4d0 legacy=0x7f6cf406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 -- 172.21.15.114:0/2620727506 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40ae710 con 0x7f6cf406c710 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf8ed5700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf40afb90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009e80 tx=0x7f6cec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b9734a8b9de1d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf2ffd700 1 -- 172.21.15.114:0/2620727506 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00e070 con 0x7f6cf406c710 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.036+0000 7f6cf2ffd700 1 -- 172.21.15.114:0/2620727506 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec003a20 con 0x7f6cf406c710 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cf2ffd700 1 -- 172.21.15.114:0/2620727506 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003dd0 con 0x7f6cf406c710 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 -- 172.21.15.114:0/2620727506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 msgr2=0x7f6cf40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf40afb90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009e80 tx=0x7f6cec00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 -- 172.21.15.114:0/2620727506 shutdown_connections 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf40afb90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2620727506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf415e080 0x7f6cf40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 -- 172.21.15.114:0/2620727506 >> 172.21.15.114:0/2620727506 conn(0x7f6cf4071b90 msgr2=0x7f6cf4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 -- 172.21.15.114:0/2620727506 shutdown_connections 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.037+0000 7f6cfa938700 1 -- 172.21.15.114:0/2620727506 wait complete. 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.038+0000 7f6cfa938700 1 Processor -- start 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.038+0000 7f6cfa938700 1 -- start start 2023-11-24T06:25:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf40af4d0 0x7f6cf4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf415e080 0x7f6cf415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4158a00 con 0x7f6cf406c710 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4158e90 con 0x7f6cf415e080 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cfa938700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4157b50 con 0x7f6cf40af4d0 2023-11-24T06:25:51.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40806/0 (socket says 172.21.15.114:40806) 2023-11-24T06:25:51.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.039+0000 7f6cf3fff700 1 -- 172.21.15.114:0/2544984403 learned_addr learned my addr 172.21.15.114:0/2544984403 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:51.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf8ed5700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf415e080 0x7f6cf415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 -- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf40af4d0 msgr2=0x7f6cf4157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.049 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf40af4d0 0x7f6cf4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 -- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf415e080 msgr2=0x7f6cf415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf415e080 0x7f6cf415bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 -- 172.21.15.114:0/2544984403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec009b30 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf3fff700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d800 tx=0x7f6ce800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80125f0 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8012750 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf4157df0 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf41583b0 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.040+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8012b00 con 0x7f6cf406c710 2023-11-24T06:25:51.050 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.041+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd40052e0 con 0x7f6cf406c710 2023-11-24T06:25:51.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.042+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6ce8012c60 con 0x7f6cf406c710 2023-11-24T06:25:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.042+0000 7f6cf0ff9700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 0x7f6ce006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.043+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce808a890 con 0x7f6cf406c710 2023-11-24T06:25:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.043+0000 7f6cf37fe700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 0x7f6ce006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.043+0000 7f6cf37fe700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 0x7f6ce006d540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cf406f220 tx=0x7f6ce4007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.046+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ce8055610 con 0x7f6cf406c710 2023-11-24T06:25:51.240 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.238+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cd4005860 con 0x7f6cf406c710 2023-11-24T06:25:51.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.239+0000 7f6cf0ff9700 1 -- 172.21.15.114:0/2544984403 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f6ce8058c30 con 0x7f6cf406c710 2023-11-24T06:25:51.241 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:51.242 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":44,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:49.912907+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 msgr2=0x7f6ce006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 0x7f6ce006d540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cf406f220 tx=0x7f6ce4007a40 comp rx=0 tx=0).stop 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 msgr2=0x7f6cf4157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d800 tx=0x7f6ce800dbc0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 shutdown_connections 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6ce006b080 0x7f6ce006d540 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cf415e080 0x7f6cf415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cf406c710 0x7f6cf4157060 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 --2- 172.21.15.114:0/2544984403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cf40af4d0 0x7f6cf4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 >> 172.21.15.114:0/2544984403 conn(0x7f6cf4071b90 msgr2=0x7f6cf40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 shutdown_connections 2023-11-24T06:25:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.241+0000 7f6cfa938700 1 -- 172.21.15.114:0/2544984403 wait complete. 2023-11-24T06:25:51.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:25:51.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.376+0000 7fe0c3561700 1 Processor -- start 2023-11-24T06:25:51.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.376+0000 7fe0c3561700 1 -- start start 2023-11-24T06:25:51.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c3561700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0bc0c6c30 0x7fe0bc0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c3561700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 0x7fe0bc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c3561700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0d0140 con 0x7fe0bc15bf50 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c3561700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc15c330 con 0x7fe0bc0c6c30 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c3561700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0cf3d0 con 0x7fe0bc0cef40 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c1afe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 0x7fe0bc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c0afc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0bc0c6c30 0x7fe0bc0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c12fd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe0bc15bf50 0x7fe0bc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:50118/0 (socket says 172.21.15.114:50118) 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.377+0000 7fe0c12fd700 1 -- 172.21.15.114:0/4150544356 learned_addr learned my addr 172.21.15.114:0/4150544356 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0c1afe700 1 -- 172.21.15.114:0/4150544356 >> v2:172.21.15.172:3300/0 conn(0x7fe0bc0c6c30 msgr2=0x7fe0bc0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0c1afe700 1 --2- 172.21.15.114:0/4150544356 >> v2:172.21.15.172:3300/0 conn(0x7fe0bc0c6c30 0x7fe0bc0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0b3fff700 1 -- 172.21.15.114:0/4150544356 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502792697 0 0) 0x7fe0bc0d0140 con 0x7fe0bc15bf50 2023-11-24T06:25:51.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0c1afe700 1 -- 172.21.15.114:0/4150544356 >> v1:172.21.15.172:6789/0 conn(0x7fe0bc15bf50 legacy=0x7fe0bc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0c1afe700 1 -- 172.21.15.114:0/4150544356 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0d03e0 con 0x7fe0bc0cef40 2023-11-24T06:25:51.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0c1afe700 1 --2- 172.21.15.114:0/4150544356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 0x7fe0bc0cc6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6528597ee3a2e25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0b3fff700 1 -- 172.21.15.114:0/4150544356 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400e070 con 0x7fe0bc0cef40 2023-11-24T06:25:51.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0b3fff700 1 -- 172.21.15.114:0/4150544356 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400bb20 con 0x7fe0bc0cef40 2023-11-24T06:25:51.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.378+0000 7fe0b3fff700 1 -- 172.21.15.114:0/4150544356 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4012430 con 0x7fe0bc0cef40 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 -- 172.21.15.114:0/4150544356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 msgr2=0x7fe0bc0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 --2- 172.21.15.114:0/4150544356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 0x7fe0bc0cc6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b400b330 comp rx=0 tx=0).stop 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 -- 172.21.15.114:0/4150544356 shutdown_connections 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 --2- 172.21.15.114:0/4150544356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc0cef40 0x7fe0bc0cc6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 --2- 172.21.15.114:0/4150544356 >> v2:172.21.15.172:3300/0 conn(0x7fe0bc0c6c30 0x7fe0bc0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 -- 172.21.15.114:0/4150544356 >> 172.21.15.114:0/4150544356 conn(0x7fe0bc0b3780 msgr2=0x7fe0bc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 -- 172.21.15.114:0/4150544356 shutdown_connections 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.379+0000 7fe0c3561700 1 -- 172.21.15.114:0/4150544356 wait complete. 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.380+0000 7fe0c3561700 1 Processor -- start 2023-11-24T06:25:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.380+0000 7fe0c3561700 1 -- start start 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0bc0c6c30 0x7fe0bc154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc15bf50 0x7fe0bc159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc156e10 con 0x7fe0bc0cef40 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0c9fc0 con 0x7fe0bc0c6c30 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc154960 con 0x7fe0bc15bf50 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40822/0 (socket says 172.21.15.114:40822) 2023-11-24T06:25:51.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 -- 172.21.15.114:0/2958664255 learned_addr learned my addr 172.21.15.114:0/2958664255 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c12fd700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0bc0c6c30 0x7fe0bc154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c1afe700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc15bf50 0x7fe0bc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 -- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc15bf50 msgr2=0x7fe0bc159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc15bf50 0x7fe0bc159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 -- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0bc0c6c30 msgr2=0x7fe0bc154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0bc0c6c30 0x7fe0bc154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 -- 172.21.15.114:0/2958664255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4009a70 con 0x7fe0bc0cef40 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c0afc700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800e9b0 tx=0x7fe0b800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8009710 con 0x7fe0bc0cef40 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800ee50 con 0x7fe0bc0cef40 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc156580 con 0x7fe0bc0cef40 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.381+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800ca50 con 0x7fe0bc0cef40 2023-11-24T06:25:51.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.382+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc156b30 con 0x7fe0bc0cef40 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.383+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc0ca860 con 0x7fe0bc0cef40 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.383+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe0b800cbb0 con 0x7fe0bc0cef40 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.383+0000 7fe0b1ffb700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 0x7fe0a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.384+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0b80898d0 con 0x7fe0bc0cef40 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.384+0000 7fe0c12fd700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 0x7fe0a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.384+0000 7fe0c12fd700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 0x7fe0a006d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac000f60 tx=0x7fe0ac007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:51.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.388+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0b8054700 con 0x7fe0bc0cef40 2023-11-24T06:25:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:51.621+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0bc068fb0 con 0x7fe0bc0cef40 2023-11-24T06:25:52.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.544+0000 7fe0b1ffb700 1 -- 172.21.15.114:0/2958664255 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0b80210b0 con 0x7fe0bc0cef40 2023-11-24T06:25:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 msgr2=0x7fe0a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 0x7fe0a006d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac000f60 tx=0x7fe0ac007e70 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 msgr2=0x7fe0bc156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800e9b0 tx=0x7fe0b800ecc0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 shutdown_connections 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe0a006b080 0x7fe0a006d540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0bc15bf50 0x7fe0bc159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0bc0c6c30 0x7fe0bc154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 --2- 172.21.15.114:0/2958664255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc0cef40 0x7fe0bc156040 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 >> 172.21.15.114:0/2958664255 conn(0x7fe0bc0b3780 msgr2=0x7fe0bc0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 shutdown_connections 2023-11-24T06:25:52.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.547+0000 7fe0c3561700 1 -- 172.21.15.114:0/2958664255 wait complete. 2023-11-24T06:25:52.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:25:52.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:25:52.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.691+0000 7fb12ac9f700 1 Processor -- start 2023-11-24T06:25:52.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.691+0000 7fb12ac9f700 1 -- start start 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb12ac9f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb1240ade90 0x7fb12406b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb12ac9f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240b5b60 con 0x7fb124145560 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb12ac9f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240b5e40 con 0x7fb124073f90 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb12ac9f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240b6120 con 0x7fb1240ade90 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb128a3b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb124145560 0x7fb1240b0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56536/0 (socket says 172.21.15.114:56536) 2023-11-24T06:25:52.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb128a3b700 1 -- 172.21.15.114:0/1669602221 learned_addr learned my addr 172.21.15.114:0/1669602221 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb123fff700 1 --2- 172.21.15.114:0/1669602221 >> v2:172.21.15.172:3300/0 conn(0x7fb1240ade90 0x7fb12406b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472420604 0 0) 0x7fb1240b5b60 con 0x7fb124145560 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1140035b0 con 0x7fb124145560 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983798444 0 0) 0x7fb1140035b0 con 0x7fb124145560 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 >> v2:172.21.15.172:3300/0 conn(0x7fb1240ade90 msgr2=0x7fb12406b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:52.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 --2- 172.21.15.114:0/1669602221 >> v2:172.21.15.172:3300/0 conn(0x7fb1240ade90 0x7fb12406b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 >> v1:172.21.15.172:6789/0 conn(0x7fb124073f90 legacy=0x7fb1240743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12406b190 con 0x7fb124145560 2023-11-24T06:25:52.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fb118001670 con 0x7fb124145560 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.692+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 >> v1:172.21.15.114:6789/0 conn(0x7fb124145560 legacy=0x7fb1240b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb114003a00 0x7fb114003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 0x7fb114004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114005210 0x7fb11400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b6120 con 0x7fb114004850 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11400c1b0 con 0x7fb114003a00 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11400c370 con 0x7fb114005210 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 0x7fb114004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb128a3b700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb114003a00 0x7fb114003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb12923c700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114005210 0x7fb11400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 -- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114005210 msgr2=0x7fb11400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114005210 0x7fb11400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 -- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb114003a00 msgr2=0x7fb114003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb114003a00 0x7fb114003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 -- 172.21.15.114:0/1669602221 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1140035b0 con 0x7fb114004850 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb12923c700 1 -- 172.21.15.114:0/1669602221 reap_dead start 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb123fff700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 0x7fb114004cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb110009b60 tx=0x7fb114005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f8e3fdaa9df033 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000e6c0 con 0x7fb114004850 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11000e820 con 0x7fb114004850 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.693+0000 7fb1237fe700 1 -- 172.21.15.114:0/1669602221 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000eb90 con 0x7fb114004850 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 msgr2=0x7fb114004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.701 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 0x7fb114004cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb110009b60 tx=0x7fb114005880 comp rx=0 tx=0).stop 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1669602221 shutdown_connections 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1669602221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb114004850 0x7fb114004cd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1669602221 >> 172.21.15.114:0/1669602221 conn(0x7fb1240b28f0 msgr2=0x7fb1240b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1669602221 shutdown_connections 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.694+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1669602221 wait complete. 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.695+0000 7fb12ac9f700 1 Processor -- start 2023-11-24T06:25:52.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.695+0000 7fb12ac9f700 1 -- start start 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114004850 0x7fb12413a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb124139910 0x7fb124139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12413aa30 con 0x7fb12415f1f0 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12413a2d0 con 0x7fb124139910 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12ac9f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12413acd0 con 0x7fb114004850 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40842/0 (socket says 172.21.15.114:40842) 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 -- 172.21.15.114:0/1197809605 learned_addr learned my addr 172.21.15.114:0/1197809605 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb128a3b700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114004850 0x7fb12413a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb12923c700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb124139910 0x7fb124139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 -- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114004850 msgr2=0x7fb12413a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114004850 0x7fb12413a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 -- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb124139910 msgr2=0x7fb124139d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb124139910 0x7fb124139d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.696+0000 7fb123fff700 1 -- 172.21.15.114:0/1197809605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12413afb0 con 0x7fb12415f1f0 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb123fff700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c180 tx=0x7fb1100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000e6c0 con 0x7fb12415f1f0 2023-11-24T06:25:52.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb110003ad0 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12413b290 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110003e80 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.697+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12413b850 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.698+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12413bbc0 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.698+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb110013470 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.698+0000 7fb1217fa700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 0x7fb10806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.698+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb11008b3b0 con 0x7fb12415f1f0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.698+0000 7fb128a3b700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 0x7fb10806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:52.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.699+0000 7fb128a3b700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 0x7fb10806d540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb124064e10 tx=0x7fb118009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:52.708 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.703+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb110056390 con 0x7fb12415f1f0 2023-11-24T06:25:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:52.938+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb124068fb0 con 0x7fb12415f1f0 2023-11-24T06:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.546+0000 7fb1217fa700 1 -- 172.21.15.114:0/1197809605 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1100599b0 con 0x7fb12415f1f0 2023-11-24T06:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 msgr2=0x7fb10806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 0x7fb10806d540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb124064e10 tx=0x7fb118009040 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 msgr2=0x7fb12413dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c180 tx=0x7fb1100047c0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 shutdown_connections 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb10806b080 0x7fb10806d540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb124139910 0x7fb124139d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb114004850 0x7fb12413a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 --2- 172.21.15.114:0/1197809605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb12415f1f0 0x7fb12413dce0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 >> 172.21.15.114:0/1197809605 conn(0x7fb1240b28f0 msgr2=0x7fb124143440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 shutdown_connections 2023-11-24T06:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.549+0000 7fb12ac9f700 1 -- 172.21.15.114:0/1197809605 wait complete. 2023-11-24T06:25:53.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:25:53.565 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:25:53.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.686+0000 7fe57629c700 1 Processor -- start 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 -- start start 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe57015e080 0x7fe5700b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57006c710 0x7fe5700afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5700ae430 con 0x7fe5700af4d0 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5700af0b0 con 0x7fe57015e080 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe57629c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700b05a0 con 0x7fe57006c710 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe56ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe57015e080 0x7fe5700b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.687+0000 7fe56ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe57015e080 0x7fe5700b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40848/0 (socket says 172.21.15.114:40848) 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 -- 172.21.15.114:0/461769184 learned_addr learned my addr 172.21.15.114:0/461769184 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 -- 172.21.15.114:0/461769184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57006c710 msgr2=0x7fe5700afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe574839700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57006c710 0x7fe5700afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57006c710 0x7fe5700afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 -- 172.21.15.114:0/461769184 >> v1:172.21.15.172:6789/0 conn(0x7fe5700af4d0 legacy=0x7fe57006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 -- 172.21.15.114:0/461769184 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700ae710 con 0x7fe57015e080 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56ffff700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57015e080 0x7fe5700b5050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe560009920 tx=0x7fe560009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f07288aa1ea9de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56effd700 1 -- 172.21.15.114:0/461769184 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560004230 con 0x7fe57015e080 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56effd700 1 -- 172.21.15.114:0/461769184 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe560004390 con 0x7fe57015e080 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.688+0000 7fe56effd700 1 -- 172.21.15.114:0/461769184 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560009e70 con 0x7fe57015e080 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 -- 172.21.15.114:0/461769184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57015e080 msgr2=0x7fe5700b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57015e080 0x7fe5700b5050 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe560009920 tx=0x7fe560009c30 comp rx=0 tx=0).stop 2023-11-24T06:25:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 -- 172.21.15.114:0/461769184 shutdown_connections 2023-11-24T06:25:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57006c710 0x7fe5700afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 --2- 172.21.15.114:0/461769184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe57015e080 0x7fe5700b5050 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 -- 172.21.15.114:0/461769184 >> 172.21.15.114:0/461769184 conn(0x7fe570071b90 msgr2=0x7fe570071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 -- 172.21.15.114:0/461769184 shutdown_connections 2023-11-24T06:25:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.689+0000 7fe57629c700 1 -- 172.21.15.114:0/461769184 wait complete. 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.690+0000 7fe57629c700 1 Processor -- start 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.690+0000 7fe57629c700 1 -- start start 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57006c710 0x7fe5701f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57015e080 0x7fe5700c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700c6180 con 0x7fe5700af4d0 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5701f4050 con 0x7fe57006c710 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe57629c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5701f4330 con 0x7fe57015e080 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe56f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe56f7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40854/0 (socket says 172.21.15.114:40854) 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.691+0000 7fe56f7fe700 1 -- 172.21.15.114:0/2759834819 learned_addr learned my addr 172.21.15.114:0/2759834819 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:53.697 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe574839700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57015e080 0x7fe5700c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56ffff700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57006c710 0x7fe5701f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 -- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57015e080 msgr2=0x7fe5700c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57015e080 0x7fe5700c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 -- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57006c710 msgr2=0x7fe5701f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57006c710 0x7fe5701f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 -- 172.21.15.114:0/2759834819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009550 con 0x7fe5700af4d0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56f7fe700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c4a0 tx=0x7fe56400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564011790 con 0x7fe5700af4d0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640118f0 con 0x7fe5700af4d0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5701f4610 con 0x7fe5700af4d0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564011ca0 con 0x7fe5700af4d0 2023-11-24T06:25:53.698 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.692+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57015e500 con 0x7fe5700af4d0 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.693+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57015e6c0 con 0x7fe5700af4d0 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.693+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe56401c460 con 0x7fe5700af4d0 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.694+0000 7fe56cff9700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 0x7fe55c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.694+0000 7fe56ffff700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 0x7fe55c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.694+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe564089750 con 0x7fe5700af4d0 2023-11-24T06:25:53.699 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.694+0000 7fe56ffff700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 0x7fe55c06d540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe560009870 tx=0x7fe560008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:53.700 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.698+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe564054580 con 0x7fe5700af4d0 2023-11-24T06:25:53.936 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.934+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5700d09b0 con 0x7fe5700af4d0 2023-11-24T06:25:53.937 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.935+0000 7fe56cff9700 1 -- 172.21.15.114:0/2759834819 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fe564057ba0 con 0x7fe5700af4d0 2023-11-24T06:25:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:53.937 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:25:53.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 msgr2=0x7fe55c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 0x7fe55c06d540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe560009870 tx=0x7fe560008030 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 msgr2=0x7fe5700c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c4a0 tx=0x7fe56400c860 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 shutdown_connections 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe55c06b080 0x7fe55c06d540 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe57015e080 0x7fe5700c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57006c710 0x7fe5701f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 --2- 172.21.15.114:0/2759834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5700af4d0 0x7fe5700c5c40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 >> 172.21.15.114:0/2759834819 conn(0x7fe570071b90 msgr2=0x7fe57006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:53.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 shutdown_connections 2023-11-24T06:25:53.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:53.937+0000 7fe57629c700 1 -- 172.21.15.114:0/2759834819 wait complete. 2023-11-24T06:25:53.941 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 27 2023-11-24T06:25:53.951 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.069+0000 7f78162d0700 1 Processor -- start 2023-11-24T06:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.069+0000 7f78162d0700 1 -- start start 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.070+0000 7f78162d0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.070+0000 7f78162d0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f781006c540 0x7f78100b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.070+0000 7f78162d0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78100af950 con 0x7f78101446a0 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.070+0000 7f78162d0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7810144a80 con 0x7f781006c540 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f78162d0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781006c9b0 con 0x7f78100adf00 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780ffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f78101446a0 0x7f78100b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56568/0 (socket says 172.21.15.114:56568) 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780ffff700 1 -- 172.21.15.114:0/526018886 learned_addr learned my addr 172.21.15.114:0/526018886 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 --2- 172.21.15.114:0/526018886 >> v2:172.21.15.114:3300/0 conn(0x7f781006c540 0x7f78100b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780f7fe700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780effd700 1 -- 172.21.15.114:0/526018886 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273045878 0 0) 0x7f78100af950 con 0x7f78101446a0 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780effd700 1 -- 172.21.15.114:0/526018886 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f80035b0 con 0x7f78101446a0 2023-11-24T06:25:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 -- 172.21.15.114:0/526018886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 msgr2=0x7f781006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 -- 172.21.15.114:0/526018886 >> v1:172.21.15.114:6789/0 conn(0x7f78101446a0 legacy=0x7f78100b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 -- 172.21.15.114:0/526018886 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100afc30 con 0x7f781006c540 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780f7fe700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f781486d700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f78100b3d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7808009dc0 tx=0x7f780800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d2e4c128073279 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780effd700 1 -- 172.21.15.114:0/526018886 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800e070 con 0x7f781006c540 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780effd700 1 -- 172.21.15.114:0/526018886 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780800bae0 con 0x7f781006c540 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.071+0000 7f780effd700 1 -- 172.21.15.114:0/526018886 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800be90 con 0x7f781006c540 2023-11-24T06:25:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 -- 172.21.15.114:0/526018886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 msgr2=0x7f78100b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f78100b3d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7808009dc0 tx=0x7f780800b330 comp rx=0 tx=0).stop 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 -- 172.21.15.114:0/526018886 shutdown_connections 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f78100b3d40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 --2- 172.21.15.114:0/526018886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 -- 172.21.15.114:0/526018886 >> 172.21.15.114:0/526018886 conn(0x7f7810071b90 msgr2=0x7f7810071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 -- 172.21.15.114:0/526018886 shutdown_connections 2023-11-24T06:25:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.072+0000 7f78162d0700 1 -- 172.21.15.114:0/526018886 wait complete. 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.073+0000 7f78162d0700 1 Processor -- start 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.073+0000 7f78162d0700 1 -- start start 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78101446a0 0x7f781015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7810139460 con 0x7f781006c540 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781015c810 con 0x7f78101446a0 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f78162d0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781015caf0 con 0x7f78100adf00 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40868/0 (socket says 172.21.15.114:40868) 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 -- 172.21.15.114:0/2846471997 learned_addr learned my addr 172.21.15.114:0/2846471997 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 -- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 msgr2=0x7f781006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 -- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78101446a0 msgr2=0x7f781015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f781486d700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78101446a0 0x7f781015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.074+0000 7f780ffff700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78101446a0 0x7f781015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f780ffff700 1 -- 172.21.15.114:0/2846471997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808009a70 con 0x7f781006c540 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f780ffff700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7800001eb0 tx=0x7f780000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800005d70 con 0x7f781006c540 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78000070d0 con 0x7f781006c540 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781015cdd0 con 0x7f781006c540 2023-11-24T06:25:54.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7800007460 con 0x7f781006c540 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.075+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781015e130 con 0x7f781006c540 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.076+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78100753f0 con 0x7f781006c540 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.076+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f78000075c0 con 0x7f781006c540 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.077+0000 7f780cff9700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 0x7f77fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.077+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7800088ee0 con 0x7f781006c540 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.077+0000 7f780f7fe700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 0x7f77fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.077+0000 7f780f7fe700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 0x7f77fc06d540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7804006fd0 tx=0x7f7804008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.081+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7800053c70 con 0x7f781006c540 2023-11-24T06:25:54.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.273+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f781015d2f0 con 0x7f781006c540 2023-11-24T06:25:54.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.274+0000 7f780cff9700 1 -- 172.21.15.114:0/2846471997 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f781015d2f0 con 0x7f781006c540 2023-11-24T06:25:54.277 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:54.277 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":45,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:51.523973+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 msgr2=0x7f77fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 0x7f77fc06d540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7804006fd0 tx=0x7f7804008040 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 msgr2=0x7f781015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7800001eb0 tx=0x7f780000c860 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 shutdown_connections 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f77fc06b080 0x7f77fc06d540 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78101446a0 0x7f781015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f781006c540 0x7f781015dbf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 --2- 172.21.15.114:0/2846471997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78100adf00 0x7f781006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 >> 172.21.15.114:0/2846471997 conn(0x7f7810071b90 msgr2=0x7f78100cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:54.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 shutdown_connections 2023-11-24T06:25:54.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.276+0000 7f78162d0700 1 -- 172.21.15.114:0/2846471997 wait complete. 2023-11-24T06:25:54.292 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:25:54.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:54.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.410+0000 7fc2e55c4700 1 Processor -- start 2023-11-24T06:25:54.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.410+0000 7fc2e55c4700 1 -- start start 2023-11-24T06:25:54.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2e55c4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2e0161660 0x7fc2e00b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2e55c4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc2e0161cf0 0x7fc2e00b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2e55c4700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e0161a40 con 0x7fc2e00b6400 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2e55c4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00b6fb0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2e55c4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00d0210 con 0x7fc2e0161660 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc2e0161cf0 0x7fc2e00b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2df7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc2e00b6400 0x7fc2e006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56584/0 (socket says 172.21.15.114:56584) 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2df7fe700 1 -- 172.21.15.114:0/4073080248 learned_addr learned my addr 172.21.15.114:0/4073080248 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2deffd700 1 --2- 172.21.15.114:0/4073080248 >> v2:172.21.15.172:3300/0 conn(0x7fc2e0161660 0x7fc2e00b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 -- 172.21.15.114:0/4073080248 >> v2:172.21.15.172:3300/0 conn(0x7fc2e0161660 msgr2=0x7fc2e00b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 --2- 172.21.15.114:0/4073080248 >> v2:172.21.15.172:3300/0 conn(0x7fc2e0161660 0x7fc2e00b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 -- 172.21.15.114:0/4073080248 >> v1:172.21.15.114:6789/0 conn(0x7fc2e00b6400 legacy=0x7fc2e006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 -- 172.21.15.114:0/4073080248 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0162540 con 0x7fc2e0161cf0 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2de7fc700 1 --2- 172.21.15.114:0/4073080248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00b5ec0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc2e008bfb0 tx=0x7fc2d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2feda9a27b7e57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2ddffb700 1 -- 172.21.15.114:0/4073080248 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011830 con 0x7fc2e0161cf0 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2ddffb700 1 -- 172.21.15.114:0/4073080248 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4011990 con 0x7fc2e0161cf0 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.411+0000 7fc2ddffb700 1 -- 172.21.15.114:0/4073080248 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011d20 con 0x7fc2e0161cf0 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 -- 172.21.15.114:0/4073080248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 msgr2=0x7fc2e00b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/4073080248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00b5ec0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc2e008bfb0 tx=0x7fc2d4009b20 comp rx=0 tx=0).stop 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 -- 172.21.15.114:0/4073080248 shutdown_connections 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/4073080248 >> v2:172.21.15.172:3300/0 conn(0x7fc2e0161660 0x7fc2e00b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/4073080248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00b5ec0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 -- 172.21.15.114:0/4073080248 >> 172.21.15.114:0/4073080248 conn(0x7fc2e00d42a0 msgr2=0x7fc2e00d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 -- 172.21.15.114:0/4073080248 shutdown_connections 2023-11-24T06:25:54.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.412+0000 7fc2e55c4700 1 -- 172.21.15.114:0/4073080248 wait complete. 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.413+0000 7fc2e55c4700 1 Processor -- start 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 -- start start 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00b6400 0x7fc2e00cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e0161660 0x7fc2e00c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015e2e0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015e5c0 con 0x7fc2e00b6400 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.414+0000 7fc2e55c4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015e8a0 con 0x7fc2e0161660 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40882/0 (socket says 172.21.15.114:40882) 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 -- 172.21.15.114:0/3250063134 learned_addr learned my addr 172.21.15.114:0/3250063134 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2deffd700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00b6400 0x7fc2e00cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2de7fc700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e0161660 0x7fc2e00c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 -- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e0161660 msgr2=0x7fc2e00c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e0161660 0x7fc2e00c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 -- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00b6400 msgr2=0x7fc2e00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00b6400 0x7fc2e00cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 -- 172.21.15.114:0/3250063134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40094a0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2df7fe700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800cbc0 tx=0x7fc2d800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800a9e0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800ab40 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e015eb80 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8012400 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.415+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e015d4b0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.416+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e00cb1a0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.417+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc2d801b3b0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.417+0000 7fc2c77fe700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 0x7fc2cc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.417+0000 7fc2deffd700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 0x7fc2cc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.417+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d80893b0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.418+0000 7fc2deffd700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 0x7fc2cc06d540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006b20 tx=0x7fc2d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.422+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2e00cb1a0 con 0x7fc2e0161cf0 2023-11-24T06:25:54.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.611+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2e0162b00 con 0x7fc2cc06b080 2023-11-24T06:25:54.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.619+0000 7fc2c77fe700 1 -- 172.21.15.114:0/3250063134 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283479 (secure 0 0 0) 0x7fc2d0009e40 con 0x7fc2cc06b080 2023-11-24T06:25:54.624 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 msgr2=0x7fc2cc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 0x7fc2cc06d540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006b20 tx=0x7fc2d0008040 comp rx=0 tx=0).stop 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 msgr2=0x7fc2e00c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800cbc0 tx=0x7fc2d800ced0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 shutdown_connections 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc2cc06b080 0x7fc2cc06d540 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2e0161cf0 0x7fc2e00c9450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00b6400 0x7fc2e00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 --2- 172.21.15.114:0/3250063134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e0161660 0x7fc2e00c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 >> 172.21.15.114:0/3250063134 conn(0x7fc2e00d42a0 msgr2=0x7fc2e015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:54.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 shutdown_connections 2023-11-24T06:25:54.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.622+0000 7fc2e55c4700 1 -- 172.21.15.114:0/3250063134 wait complete. 2023-11-24T06:25:54.628 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:54.637 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-24T06:25:53.725877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221188,"kb_used_data":5012,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753532,"statfs":{"total":773094113280,"available":772867616768,"internally_reserved":0,"allocated":5132288,"data_stored":2747843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220180"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515714+0000","last_change":"2023-11-24T06:25:49.515714+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515714+0000","last_undegraded":"2023-11-24T06:25:49.515714+0000","last_fullsized":"2023-11-24T06:25:49.515714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522707+0000","last_change":"2023-11-24T06:25:49.522707+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522707+0000","last_undegraded":"2023-11-24T06:25:49.522707+0000","last_fullsized":"2023-11-24T06:25:49.522707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513933+0000","last_change":"2023-11-24T06:25:49.513933+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513933+0000","last_undegraded":"2023-11-24T06:25:49.513933+0000","last_fullsized":"2023-11-24T06:25:49.513933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521392+0000","last_change":"2023-11-24T06:25:49.521392+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521392+0000","last_undegraded":"2023-11-24T06:25:49.521392+0000","last_fullsized":"2023-11-24T06:25:49.521392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522718+0000","last_change":"2023-11-24T06:25:49.522718+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522718+0000","last_undegraded":"2023-11-24T06:25:49.522718+0000","last_fullsized":"2023-11-24T06:25:49.522718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526883+0000","last_change":"2023-11-24T06:25:49.526883+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526883+0000","last_undegraded":"2023-11-24T06:25:49.526883+0000","last_fullsized":"2023-11-24T06:25:49.526883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516210+0000","last_change":"2023-11-24T06:25:49.516210+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516210+0000","last_undegraded":"2023-11-24T06:25:49.516210+0000","last_fullsized":"2023-11-24T06:25:49.516210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521856+0000","last_change":"2023-11-24T06:25:49.521856+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521856+0000","last_undegraded":"2023-11-24T06:25:49.521856+0000","last_fullsized":"2023-11-24T06:25:49.521856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520072+0000","last_change":"2023-11-24T06:25:49.520072+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520072+0000","last_undegraded":"2023-11-24T06:25:49.520072+0000","last_fullsized":"2023-11-24T06:25:49.520072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520473+0000","last_change":"2023-11-24T06:25:49.520473+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520473+0000","last_undegraded":"2023-11-24T06:25:49.520473+0000","last_fullsized":"2023-11-24T06:25:49.520473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519995+0000","last_change":"2023-11-24T06:25:49.519995+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519995+0000","last_undegraded":"2023-11-24T06:25:49.519995+0000","last_fullsized":"2023-11-24T06:25:49.519995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524278+0000","last_change":"2023-11-24T06:25:49.524278+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524278+0000","last_undegraded":"2023-11-24T06:25:49.524278+0000","last_fullsized":"2023-11-24T06:25:49.524278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517000+0000","last_change":"2023-11-24T06:25:49.517000+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517000+0000","last_undegraded":"2023-11-24T06:25:49.517000+0000","last_fullsized":"2023-11-24T06:25:49.517000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520005+0000","last_change":"2023-11-24T06:25:49.520005+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520005+0000","last_undegraded":"2023-11-24T06:25:49.520005+0000","last_fullsized":"2023-11-24T06:25:49.520005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525682+0000","last_change":"2023-11-24T06:25:49.525682+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525682+0000","last_undegraded":"2023-11-24T06:25:49.525682+0000","last_fullsized":"2023-11-24T06:25:49.525682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517415+0000","last_change":"2023-11-24T06:25:49.517415+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517415+0000","last_undegraded":"2023-11-24T06:25:49.517415+0000","last_fullsized":"2023-11-24T06:25:49.517415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513815+0000","last_change":"2023-11-24T06:25:49.513815+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513815+0000","last_undegraded":"2023-11-24T06:25:49.513815+0000","last_fullsized":"2023-11-24T06:25:49.513815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515328+0000","last_change":"2023-11-24T06:25:49.515328+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515328+0000","last_undegraded":"2023-11-24T06:25:49.515328+0000","last_fullsized":"2023-11-24T06:25:49.515328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518515+0000","last_change":"2023-11-24T06:25:49.518515+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518515+0000","last_undegraded":"2023-11-24T06:25:49.518515+0000","last_fullsized":"2023-11-24T06:25:49.518515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516635+0000","last_change":"2023-11-24T06:25:49.516635+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516635+0000","last_undegraded":"2023-11-24T06:25:49.516635+0000","last_fullsized":"2023-11-24T06:25:49.516635+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.527437+0000","last_change":"2023-11-24T06:25:49.527437+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.527437+0000","last_undegraded":"2023-11-24T06:25:49.527437+0000","last_fullsized":"2023-11-24T06:25:49.527437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519517+0000","last_change":"2023-11-24T06:25:49.519517+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519517+0000","last_undegraded":"2023-11-24T06:25:49.519517+0000","last_fullsized":"2023-11-24T06:25:49.519517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518630+0000","last_change":"2023-11-24T06:25:49.518630+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518630+0000","last_undegraded":"2023-11-24T06:25:49.518630+0000","last_fullsized":"2023-11-24T06:25:49.518630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516923+0000","last_change":"2023-11-24T06:25:49.516923+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516923+0000","last_undegraded":"2023-11-24T06:25:49.516923+0000","last_fullsized":"2023-11-24T06:25:49.516923+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517579+0000","last_change":"2023-11-24T06:25:49.517579+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517579+0000","last_undegraded":"2023-11-24T06:25:49.517579+0000","last_fullsized":"2023-11-24T06:25:49.517579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516556+0000","last_change":"2023-11-24T06:25:49.516556+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516556+0000","last_undegraded":"2023-11-24T06:25:49.516556+0000","last_fullsized":"2023-11-24T06:25:49.516556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513365+0000","last_change":"2023-11-24T06:25:49.513365+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513365+0000","last_undegraded":"2023-11-24T06:25:49.513365+0000","last_fullsized":"2023-11-24T06:25:49.513365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525130+0000","last_change":"2023-11-24T06:25:49.525130+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525130+0000","last_undegraded":"2023-11-24T06:25:49.525130+0000","last_fullsized":"2023-11-24T06:25:49.525130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516966+0000","last_change":"2023-11-24T06:25:49.516966+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516966+0000","last_undegraded":"2023-11-24T06:25:49.516966+0000","last_fullsized":"2023-11-24T06:25:49.516966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.514385+0000","last_change":"2023-11-24T06:25:49.514385+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.514385+0000","last_undegraded":"2023-11-24T06:25:49.514385+0000","last_fullsized":"2023-11-24T06:25:49.514385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525215+0000","last_change":"2023-11-24T06:25:49.525215+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525215+0000","last_undegraded":"2023-11-24T06:25:49.525215+0000","last_fullsized":"2023-11-24T06:25:49.525215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516434+0000","last_change":"2023-11-24T06:25:49.516434+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516434+0000","last_undegraded":"2023-11-24T06:25:49.516434+0000","last_fullsized":"2023-11-24T06:25:49.516434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519016+0000","last_change":"2023-11-24T06:25:49.519016+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519016+0000","last_undegraded":"2023-11-24T06:25:49.519016+0000","last_fullsized":"2023-11-24T06:25:49.519016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534732+0000","last_change":"2023-11-24T06:25:49.531680+0000","last_active":"2023-11-24T06:25:49.534732+0000","last_peered":"2023-11-24T06:25:49.534732+0000","last_clean":"2023-11-24T06:25:49.534732+0000","last_became_active":"2023-11-24T06:25:49.531361+0000","last_became_peered":"2023-11-24T06:25:49.531361+0000","last_unstale":"2023-11-24T06:25:49.534732+0000","last_undegraded":"2023-11-24T06:25:49.534732+0000","last_fullsized":"2023-11-24T06:25:49.534732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519849+0000","last_change":"2023-11-24T06:25:49.519849+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519849+0000","last_undegraded":"2023-11-24T06:25:49.519849+0000","last_fullsized":"2023-11-24T06:25:49.519849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528570+0000","last_change":"2023-11-24T06:25:49.525389+0000","last_active":"2023-11-24T06:25:49.528570+0000","last_peered":"2023-11-24T06:25:49.528570+0000","last_clean":"2023-11-24T06:25:49.528570+0000","last_became_active":"2023-11-24T06:25:49.524856+0000","last_became_peered":"2023-11-24T06:25:49.524856+0000","last_unstale":"2023-11-24T06:25:49.528570+0000","last_undegraded":"2023-11-24T06:25:49.528570+0000","last_fullsized":"2023-11-24T06:25:49.528570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:11.371497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524684+0000","last_change":"2023-11-24T06:25:49.524684+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524684+0000","last_undegraded":"2023-11-24T06:25:49.524684+0000","last_fullsized":"2023-11-24T06:25:49.524684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534919+0000","last_change":"2023-11-24T06:25:49.530053+0000","last_active":"2023-11-24T06:25:49.534919+0000","last_peered":"2023-11-24T06:25:49.534919+0000","last_clean":"2023-11-24T06:25:49.534919+0000","last_became_active":"2023-11-24T06:25:49.529544+0000","last_became_peered":"2023-11-24T06:25:49.529544+0000","last_unstale":"2023-11-24T06:25:49.534919+0000","last_undegraded":"2023-11-24T06:25:49.534919+0000","last_fullsized":"2023-11-24T06:25:49.534919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:21.180992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518734+0000","last_change":"2023-11-24T06:25:49.518734+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518734+0000","last_undegraded":"2023-11-24T06:25:49.518734+0000","last_fullsized":"2023-11-24T06:25:49.518734+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531177+0000","last_change":"2023-11-24T06:25:49.529413+0000","last_active":"2023-11-24T06:25:49.531177+0000","last_peered":"2023-11-24T06:25:49.531177+0000","last_clean":"2023-11-24T06:25:49.531177+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.531177+0000","last_undegraded":"2023-11-24T06:25:49.531177+0000","last_fullsized":"2023-11-24T06:25:49.531177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513919+0000","last_change":"2023-11-24T06:25:49.513919+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513919+0000","last_undegraded":"2023-11-24T06:25:49.513919+0000","last_fullsized":"2023-11-24T06:25:49.513919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537262+0000","last_change":"2023-11-24T06:25:49.532801+0000","last_active":"2023-11-24T06:25:49.537262+0000","last_peered":"2023-11-24T06:25:49.537262+0000","last_clean":"2023-11-24T06:25:49.537262+0000","last_became_active":"2023-11-24T06:25:49.531990+0000","last_became_peered":"2023-11-24T06:25:49.531990+0000","last_unstale":"2023-11-24T06:25:49.537262+0000","last_undegraded":"2023-11-24T06:25:49.537262+0000","last_fullsized":"2023-11-24T06:25:49.537262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:58.851781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520725+0000","last_change":"2023-11-24T06:25:49.520725+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520725+0000","last_undegraded":"2023-11-24T06:25:49.520725+0000","last_fullsized":"2023-11-24T06:25:49.520725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535363+0000","last_change":"2023-11-24T06:25:49.529122+0000","last_active":"2023-11-24T06:25:49.535363+0000","last_peered":"2023-11-24T06:25:49.535363+0000","last_clean":"2023-11-24T06:25:49.535363+0000","last_became_active":"2023-11-24T06:25:49.528854+0000","last_became_peered":"2023-11-24T06:25:49.528854+0000","last_unstale":"2023-11-24T06:25:49.535363+0000","last_undegraded":"2023-11-24T06:25:49.535363+0000","last_fullsized":"2023-11-24T06:25:49.535363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:41.758296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519532+0000","last_change":"2023-11-24T06:25:49.519532+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519532+0000","last_undegraded":"2023-11-24T06:25:49.519532+0000","last_fullsized":"2023-11-24T06:25:49.519532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535377+0000","last_change":"2023-11-24T06:25:49.529495+0000","last_active":"2023-11-24T06:25:49.535377+0000","last_peered":"2023-11-24T06:25:49.535377+0000","last_clean":"2023-11-24T06:25:49.535377+0000","last_became_active":"2023-11-24T06:25:49.529256+0000","last_became_peered":"2023-11-24T06:25:49.529256+0000","last_unstale":"2023-11-24T06:25:49.535377+0000","last_undegraded":"2023-11-24T06:25:49.535377+0000","last_fullsized":"2023-11-24T06:25:49.535377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517845+0000","last_change":"2023-11-24T06:25:49.517845+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517845+0000","last_undegraded":"2023-11-24T06:25:49.517845+0000","last_fullsized":"2023-11-24T06:25:49.517845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.527150+0000","last_change":"2023-11-24T06:25:49.522891+0000","last_active":"2023-11-24T06:25:49.527150+0000","last_peered":"2023-11-24T06:25:49.527150+0000","last_clean":"2023-11-24T06:25:49.527150+0000","last_became_active":"2023-11-24T06:25:49.522605+0000","last_became_peered":"2023-11-24T06:25:49.522605+0000","last_unstale":"2023-11-24T06:25:49.527150+0000","last_undegraded":"2023-11-24T06:25:49.527150+0000","last_fullsized":"2023-11-24T06:25:49.527150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:54.895968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513631+0000","last_change":"2023-11-24T06:25:49.513631+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513631+0000","last_undegraded":"2023-11-24T06:25:49.513631+0000","last_fullsized":"2023-11-24T06:25:49.513631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525820+0000","last_change":"2023-11-24T06:25:49.522358+0000","last_active":"2023-11-24T06:25:49.525820+0000","last_peered":"2023-11-24T06:25:49.525820+0000","last_clean":"2023-11-24T06:25:49.525820+0000","last_became_active":"2023-11-24T06:25:49.521982+0000","last_became_peered":"2023-11-24T06:25:49.521982+0000","last_unstale":"2023-11-24T06:25:49.525820+0000","last_undegraded":"2023-11-24T06:25:49.525820+0000","last_fullsized":"2023-11-24T06:25:49.525820+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:36.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515185+0000","last_change":"2023-11-24T06:25:49.515185+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515185+0000","last_undegraded":"2023-11-24T06:25:49.515185+0000","last_fullsized":"2023-11-24T06:25:49.515185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528519+0000","last_change":"2023-11-24T06:25:49.524636+0000","last_active":"2023-11-24T06:25:49.528519+0000","last_peered":"2023-11-24T06:25:49.528519+0000","last_clean":"2023-11-24T06:25:49.528519+0000","last_became_active":"2023-11-24T06:25:49.524351+0000","last_became_peered":"2023-11-24T06:25:49.524351+0000","last_unstale":"2023-11-24T06:25:49.528519+0000","last_undegraded":"2023-11-24T06:25:49.528519+0000","last_fullsized":"2023-11-24T06:25:49.528519+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:32.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525547+0000","last_change":"2023-11-24T06:25:49.525547+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525547+0000","last_undegraded":"2023-11-24T06:25:49.525547+0000","last_fullsized":"2023-11-24T06:25:49.525547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.530358+0000","last_change":"2023-11-24T06:25:49.525281+0000","last_active":"2023-11-24T06:25:49.530358+0000","last_peered":"2023-11-24T06:25:49.530358+0000","last_clean":"2023-11-24T06:25:49.530358+0000","last_became_active":"2023-11-24T06:25:49.524899+0000","last_became_peered":"2023-11-24T06:25:49.524899+0000","last_unstale":"2023-11-24T06:25:49.530358+0000","last_undegraded":"2023-11-24T06:25:49.530358+0000","last_fullsized":"2023-11-24T06:25:49.530358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515945+0000","last_change":"2023-11-24T06:25:49.515945+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515945+0000","last_undegraded":"2023-11-24T06:25:49.515945+0000","last_fullsized":"2023-11-24T06:25:49.515945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.523174+0000","last_change":"2023-11-24T06:25:49.521264+0000","last_active":"2023-11-24T06:25:49.523174+0000","last_peered":"2023-11-24T06:25:49.523174+0000","last_clean":"2023-11-24T06:25:49.523174+0000","last_became_active":"2023-11-24T06:25:49.520985+0000","last_became_peered":"2023-11-24T06:25:49.520985+0000","last_unstale":"2023-11-24T06:25:49.523174+0000","last_undegraded":"2023-11-24T06:25:49.523174+0000","last_fullsized":"2023-11-24T06:25:49.523174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:51:01.319101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517033+0000","last_change":"2023-11-24T06:25:49.517033+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517033+0000","last_undegraded":"2023-11-24T06:25:49.517033+0000","last_fullsized":"2023-11-24T06:25:49.517033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532909+0000","last_change":"2023-11-24T06:25:49.531422+0000","last_active":"2023-11-24T06:25:49.532909+0000","last_peered":"2023-11-24T06:25:49.532909+0000","last_clean":"2023-11-24T06:25:49.532909+0000","last_became_active":"2023-11-24T06:25:49.531079+0000","last_became_peered":"2023-11-24T06:25:49.531079+0000","last_unstale":"2023-11-24T06:25:49.532909+0000","last_undegraded":"2023-11-24T06:25:49.532909+0000","last_fullsized":"2023-11-24T06:25:49.532909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:50.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520241+0000","last_change":"2023-11-24T06:25:49.520241+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520241+0000","last_undegraded":"2023-11-24T06:25:49.520241+0000","last_fullsized":"2023-11-24T06:25:49.520241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535343+0000","last_change":"2023-11-24T06:25:49.531424+0000","last_active":"2023-11-24T06:25:49.535343+0000","last_peered":"2023-11-24T06:25:49.535343+0000","last_clean":"2023-11-24T06:25:49.535343+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535343+0000","last_undegraded":"2023-11-24T06:25:49.535343+0000","last_fullsized":"2023-11-24T06:25:49.535343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524855+0000","last_change":"2023-11-24T06:25:49.524855+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524855+0000","last_undegraded":"2023-11-24T06:25:49.524855+0000","last_fullsized":"2023-11-24T06:25:49.524855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535002+0000","last_change":"2023-11-24T06:25:49.531559+0000","last_active":"2023-11-24T06:25:49.535002+0000","last_peered":"2023-11-24T06:25:49.535002+0000","last_clean":"2023-11-24T06:25:49.535002+0000","last_became_active":"2023-11-24T06:25:49.531127+0000","last_became_peered":"2023-11-24T06:25:49.531127+0000","last_unstale":"2023-11-24T06:25:49.535002+0000","last_undegraded":"2023-11-24T06:25:49.535002+0000","last_fullsized":"2023-11-24T06:25:49.535002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:05.025358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515749+0000","last_change":"2023-11-24T06:25:49.515749+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515749+0000","last_undegraded":"2023-11-24T06:25:49.515749+0000","last_fullsized":"2023-11-24T06:25:49.515749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.533260+0000","last_change":"2023-11-24T06:25:49.529384+0000","last_active":"2023-11-24T06:25:49.533260+0000","last_peered":"2023-11-24T06:25:49.533260+0000","last_clean":"2023-11-24T06:25:49.533260+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.533260+0000","last_undegraded":"2023-11-24T06:25:49.533260+0000","last_fullsized":"2023-11-24T06:25:49.533260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515868+0000","last_change":"2023-11-24T06:25:49.515868+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515868+0000","last_undegraded":"2023-11-24T06:25:49.515868+0000","last_fullsized":"2023-11-24T06:25:49.515868+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.538150+0000","last_change":"2023-11-24T06:25:49.533140+0000","last_active":"2023-11-24T06:25:49.538150+0000","last_peered":"2023-11-24T06:25:49.538150+0000","last_clean":"2023-11-24T06:25:49.538150+0000","last_became_active":"2023-11-24T06:25:49.532965+0000","last_became_peered":"2023-11-24T06:25:49.532965+0000","last_unstale":"2023-11-24T06:25:49.538150+0000","last_undegraded":"2023-11-24T06:25:49.538150+0000","last_fullsized":"2023-11-24T06:25:49.538150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:45.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518386+0000","last_change":"2023-11-24T06:25:49.518386+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518386+0000","last_undegraded":"2023-11-24T06:25:49.518386+0000","last_fullsized":"2023-11-24T06:25:49.518386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525822+0000","last_change":"2023-11-24T06:25:49.522374+0000","last_active":"2023-11-24T06:25:49.525822+0000","last_peered":"2023-11-24T06:25:49.525822+0000","last_clean":"2023-11-24T06:25:49.525822+0000","last_became_active":"2023-11-24T06:25:49.522053+0000","last_became_peered":"2023-11-24T06:25:49.522053+0000","last_unstale":"2023-11-24T06:25:49.525822+0000","last_undegraded":"2023-11-24T06:25:49.525822+0000","last_fullsized":"2023-11-24T06:25:49.525822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:55.240572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517595+0000","last_change":"2023-11-24T06:25:49.517595+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517595+0000","last_undegraded":"2023-11-24T06:25:49.517595+0000","last_fullsized":"2023-11-24T06:25:49.517595+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535288+0000","last_change":"2023-11-24T06:25:49.528898+0000","last_active":"2023-11-24T06:25:49.535288+0000","last_peered":"2023-11-24T06:25:49.535288+0000","last_clean":"2023-11-24T06:25:49.535288+0000","last_became_active":"2023-11-24T06:25:49.528642+0000","last_became_peered":"2023-11-24T06:25:49.528642+0000","last_unstale":"2023-11-24T06:25:49.535288+0000","last_undegraded":"2023-11-24T06:25:49.535288+0000","last_fullsized":"2023-11-24T06:25:49.535288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518084+0000","last_change":"2023-11-24T06:25:49.518084+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518084+0000","last_undegraded":"2023-11-24T06:25:49.518084+0000","last_fullsized":"2023-11-24T06:25:49.518084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534539+0000","last_change":"2023-11-24T06:25:49.531568+0000","last_active":"2023-11-24T06:25:49.534539+0000","last_peered":"2023-11-24T06:25:49.534539+0000","last_clean":"2023-11-24T06:25:49.534539+0000","last_became_active":"2023-11-24T06:25:49.531275+0000","last_became_peered":"2023-11-24T06:25:49.531275+0000","last_unstale":"2023-11-24T06:25:49.534539+0000","last_undegraded":"2023-11-24T06:25:49.534539+0000","last_fullsized":"2023-11-24T06:25:49.534539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525610+0000","last_change":"2023-11-24T06:25:49.525610+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525610+0000","last_undegraded":"2023-11-24T06:25:49.525610+0000","last_fullsized":"2023-11-24T06:25:49.525610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535112+0000","last_change":"2023-11-24T06:25:49.531979+0000","last_active":"2023-11-24T06:25:49.535112+0000","last_peered":"2023-11-24T06:25:49.535112+0000","last_clean":"2023-11-24T06:25:49.535112+0000","last_became_active":"2023-11-24T06:25:49.531699+0000","last_became_peered":"2023-11-24T06:25:49.531699+0000","last_unstale":"2023-11-24T06:25:49.535112+0000","last_undegraded":"2023-11-24T06:25:49.535112+0000","last_fullsized":"2023-11-24T06:25:49.535112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518503+0000","last_change":"2023-11-24T06:25:49.518503+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518503+0000","last_undegraded":"2023-11-24T06:25:49.518503+0000","last_fullsized":"2023-11-24T06:25:49.518503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535234+0000","last_change":"2023-11-24T06:25:49.528142+0000","last_active":"2023-11-24T06:25:49.535234+0000","last_peered":"2023-11-24T06:25:49.535234+0000","last_clean":"2023-11-24T06:25:49.535234+0000","last_became_active":"2023-11-24T06:25:49.527836+0000","last_became_peered":"2023-11-24T06:25:49.527836+0000","last_unstale":"2023-11-24T06:25:49.535234+0000","last_undegraded":"2023-11-24T06:25:49.535234+0000","last_fullsized":"2023-11-24T06:25:49.535234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515808+0000","last_change":"2023-11-24T06:25:49.515808+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515808+0000","last_undegraded":"2023-11-24T06:25:49.515808+0000","last_fullsized":"2023-11-24T06:25:49.515808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528512+0000","last_change":"2023-11-24T06:25:49.524448+0000","last_active":"2023-11-24T06:25:49.528512+0000","last_peered":"2023-11-24T06:25:49.528512+0000","last_clean":"2023-11-24T06:25:49.528512+0000","last_became_active":"2023-11-24T06:25:49.524159+0000","last_became_peered":"2023-11-24T06:25:49.524159+0000","last_unstale":"2023-11-24T06:25:49.528512+0000","last_undegraded":"2023-11-24T06:25:49.528512+0000","last_fullsized":"2023-11-24T06:25:49.528512+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:42.324117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518778+0000","last_change":"2023-11-24T06:25:49.518778+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518778+0000","last_undegraded":"2023-11-24T06:25:49.518778+0000","last_fullsized":"2023-11-24T06:25:49.518778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535306+0000","last_change":"2023-11-24T06:25:49.531285+0000","last_active":"2023-11-24T06:25:49.535306+0000","last_peered":"2023-11-24T06:25:49.535306+0000","last_clean":"2023-11-24T06:25:49.535306+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535306+0000","last_undegraded":"2023-11-24T06:25:49.535306+0000","last_fullsized":"2023-11-24T06:25:49.535306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518028+0000","last_change":"2023-11-24T06:25:49.518028+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518028+0000","last_undegraded":"2023-11-24T06:25:49.518028+0000","last_fullsized":"2023-11-24T06:25:49.518028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537271+0000","last_change":"2023-11-24T06:25:49.534058+0000","last_active":"2023-11-24T06:25:49.537271+0000","last_peered":"2023-11-24T06:25:49.537271+0000","last_clean":"2023-11-24T06:25:49.537271+0000","last_became_active":"2023-11-24T06:25:49.533728+0000","last_became_peered":"2023-11-24T06:25:49.533728+0000","last_unstale":"2023-11-24T06:25:49.537271+0000","last_undegraded":"2023-11-24T06:25:49.537271+0000","last_fullsized":"2023-11-24T06:25:49.537271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517885+0000","last_change":"2023-11-24T06:25:49.517885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517885+0000","last_undegraded":"2023-11-24T06:25:49.517885+0000","last_fullsized":"2023-11-24T06:25:49.517885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531262+0000","last_change":"2023-11-24T06:25:49.526982+0000","last_active":"2023-11-24T06:25:49.531262+0000","last_peered":"2023-11-24T06:25:49.531262+0000","last_clean":"2023-11-24T06:25:49.531262+0000","last_became_active":"2023-11-24T06:25:49.526688+0000","last_became_peered":"2023-11-24T06:25:49.526688+0000","last_unstale":"2023-11-24T06:25:49.531262+0000","last_undegraded":"2023-11-24T06:25:49.531262+0000","last_fullsized":"2023-11-24T06:25:49.531262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515368+0000","last_change":"2023-11-24T06:25:49.515368+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515368+0000","last_undegraded":"2023-11-24T06:25:49.515368+0000","last_fullsized":"2023-11-24T06:25:49.515368+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532904+0000","last_change":"2023-11-24T06:25:49.529663+0000","last_active":"2023-11-24T06:25:49.532904+0000","last_peered":"2023-11-24T06:25:49.532904+0000","last_clean":"2023-11-24T06:25:49.532904+0000","last_became_active":"2023-11-24T06:25:49.529401+0000","last_became_peered":"2023-11-24T06:25:49.529401+0000","last_unstale":"2023-11-24T06:25:49.532904+0000","last_undegraded":"2023-11-24T06:25:49.532904+0000","last_fullsized":"2023-11-24T06:25:49.532904+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:52.008057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524885+0000","last_change":"2023-11-24T06:25:49.524885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524885+0000","last_undegraded":"2023-11-24T06:25:49.524885+0000","last_fullsized":"2023-11-24T06:25:49.524885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531122+0000","last_change":"2023-11-24T06:25:49.525526+0000","last_active":"2023-11-24T06:25:49.531122+0000","last_peered":"2023-11-24T06:25:49.531122+0000","last_clean":"2023-11-24T06:25:49.531122+0000","last_became_active":"2023-11-24T06:25:49.525235+0000","last_became_peered":"2023-11-24T06:25:49.525235+0000","last_unstale":"2023-11-24T06:25:49.531122+0000","last_undegraded":"2023-11-24T06:25:49.531122+0000","last_fullsized":"2023-11-24T06:25:49.531122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526406+0000","last_change":"2023-11-24T06:25:49.526406+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526406+0000","last_undegraded":"2023-11-24T06:25:49.526406+0000","last_fullsized":"2023-11-24T06:25:49.526406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.522217+0000","last_change":"2023-11-24T06:25:49.520031+0000","last_active":"2023-11-24T06:25:49.522217+0000","last_peered":"2023-11-24T06:25:49.522217+0000","last_clean":"2023-11-24T06:25:49.522217+0000","last_became_active":"2023-11-24T06:25:49.519789+0000","last_became_peered":"2023-11-24T06:25:49.519789+0000","last_unstale":"2023-11-24T06:25:49.522217+0000","last_undegraded":"2023-11-24T06:25:49.522217+0000","last_fullsized":"2023-11-24T06:25:49.522217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.523607+0000","last_change":"2023-11-24T06:25:49.523607+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.523607+0000","last_undegraded":"2023-11-24T06:25:49.523607+0000","last_fullsized":"2023-11-24T06:25:49.523607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528237+0000","last_change":"2023-11-24T06:25:49.525157+0000","last_active":"2023-11-24T06:25:49.528237+0000","last_peered":"2023-11-24T06:25:49.528237+0000","last_clean":"2023-11-24T06:25:49.528237+0000","last_became_active":"2023-11-24T06:25:49.524896+0000","last_became_peered":"2023-11-24T06:25:49.524896+0000","last_unstale":"2023-11-24T06:25:49.528237+0000","last_undegraded":"2023-11-24T06:25:49.528237+0000","last_fullsized":"2023-11-24T06:25:49.528237+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513176+0000","last_change":"2023-11-24T06:25:49.513176+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513176+0000","last_undegraded":"2023-11-24T06:25:49.513176+0000","last_fullsized":"2023-11-24T06:25:49.513176+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.529913+0000","last_change":"2023-11-24T06:25:49.528916+0000","last_active":"2023-11-24T06:25:49.529913+0000","last_peered":"2023-11-24T06:25:49.529913+0000","last_clean":"2023-11-24T06:25:49.529913+0000","last_became_active":"2023-11-24T06:25:49.528145+0000","last_became_peered":"2023-11-24T06:25:49.528145+0000","last_unstale":"2023-11-24T06:25:49.529913+0000","last_undegraded":"2023-11-24T06:25:49.529913+0000","last_fullsized":"2023-11-24T06:25:49.529913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:18.021500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517915+0000","last_change":"2023-11-24T06:25:49.517915+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517915+0000","last_undegraded":"2023-11-24T06:25:49.517915+0000","last_fullsized":"2023-11-24T06:25:49.517915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537908+0000","last_change":"2023-11-24T06:25:49.533010+0000","last_active":"2023-11-24T06:25:49.537908+0000","last_peered":"2023-11-24T06:25:49.537908+0000","last_clean":"2023-11-24T06:25:49.537908+0000","last_became_active":"2023-11-24T06:25:49.532766+0000","last_became_peered":"2023-11-24T06:25:49.532766+0000","last_unstale":"2023-11-24T06:25:49.537908+0000","last_undegraded":"2023-11-24T06:25:49.537908+0000","last_fullsized":"2023-11-24T06:25:49.537908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-24T06:25:48.509286+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:48.509286+0000","last_peered":"2023-11-24T06:25:48.509286+0000","last_clean":"2023-11-24T06:25:48.509286+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:48.509286+0000","last_undegraded":"2023-11-24T06:25:48.509286+0000","last_fullsized":"2023-11-24T06:25:48.509286+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.495}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.439},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.747}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.315}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.371}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.747}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.379}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.752}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.89400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.499},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.51100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.441},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:54.642 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.795+0000 7eff77f16700 1 Processor -- start 2023-11-24T06:25:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.795+0000 7eff77f16700 1 -- start start 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff77f16700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7eff700b0f80 0x7eff7015a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff77f16700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff700b2bd0 con 0x7eff700cd770 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff77f16700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff700b1fc0 con 0x7eff700b0ba0 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff77f16700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff700b22a0 con 0x7eff700b0f80 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff754b1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7eff700b0f80 0x7eff7015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff764b3700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7eff700cd770 0x7eff7015a980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56598/0 (socket says 172.21.15.114:56598) 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.796+0000 7eff764b3700 1 -- 172.21.15.114:0/2140980498 learned_addr learned my addr 172.21.15.114:0/2140980498 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:54.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff754b1700 1 -- 172.21.15.114:0/2140980498 >> v1:172.21.15.172:6789/0 conn(0x7eff700b0ba0 legacy=0x7eff700af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff754b1700 1 -- 172.21.15.114:0/2140980498 >> v1:172.21.15.114:6789/0 conn(0x7eff700cd770 legacy=0x7eff7015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff74cb0700 1 -- 172.21.15.114:0/2140980498 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046345158 0 0) 0x7eff700b2bd0 con 0x7eff700cd770 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff754b1700 1 -- 172.21.15.114:0/2140980498 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff700b2580 con 0x7eff700b0f80 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff754b1700 1 --2- 172.21.15.114:0/2140980498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0f80 0x7eff7015a440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009840 tx=0x7eff6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de76cb3f185bb38a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff74cb0700 1 -- 172.21.15.114:0/2140980498 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c004230 con 0x7eff700b0f80 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff74cb0700 1 -- 172.21.15.114:0/2140980498 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6c004390 con 0x7eff700b0f80 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff74cb0700 1 -- 172.21.15.114:0/2140980498 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0036a0 con 0x7eff700b0f80 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff77f16700 1 -- 172.21.15.114:0/2140980498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0f80 msgr2=0x7eff7015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.797+0000 7eff77f16700 1 --2- 172.21.15.114:0/2140980498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0f80 0x7eff7015a440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009840 tx=0x7eff6c009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.798+0000 7eff77f16700 1 -- 172.21.15.114:0/2140980498 shutdown_connections 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.798+0000 7eff77f16700 1 --2- 172.21.15.114:0/2140980498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0f80 0x7eff7015a440 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.798+0000 7eff77f16700 1 -- 172.21.15.114:0/2140980498 >> 172.21.15.114:0/2140980498 conn(0x7eff700b57b0 msgr2=0x7eff700b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.798+0000 7eff77f16700 1 -- 172.21.15.114:0/2140980498 shutdown_connections 2023-11-24T06:25:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.798+0000 7eff77f16700 1 -- 172.21.15.114:0/2140980498 wait complete. 2023-11-24T06:25:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.799+0000 7eff77f16700 1 Processor -- start 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.799+0000 7eff77f16700 1 -- start start 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.799+0000 7eff77f16700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff700b0f80 0x7eff7009c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff700cd770 0x7eff7009c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7009b7d0 con 0x7eff700b0ba0 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7009d3e0 con 0x7eff700cd770 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7009af30 con 0x7eff700b0f80 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40892/0 (socket says 172.21.15.114:40892) 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 -- 172.21.15.114:0/1059486943 learned_addr learned my addr 172.21.15.114:0/1059486943 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff754b1700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff700b0f80 0x7eff7009c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff764b3700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff700cd770 0x7eff7009c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 -- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff700b0f80 msgr2=0x7eff7009c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff700b0f80 0x7eff7009c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 -- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff700cd770 msgr2=0x7eff7009c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff700cd770 0x7eff7009c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 -- 172.21.15.114:0/1059486943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0094a0 con 0x7eff700b0ba0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff75cb2700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7eff6000c4a0 tx=0x7eff6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600054e0 con 0x7eff700b0ba0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff60003680 con 0x7eff700b0ba0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60003a30 con 0x7eff700b0ba0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7009b210 con 0x7eff700b0ba0 2023-11-24T06:25:54.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.800+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7014e9c0 con 0x7eff700b0ba0 2023-11-24T06:25:54.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.801+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7014eb50 con 0x7eff700b0ba0 2023-11-24T06:25:54.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.802+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7eff60003b90 con 0x7eff700b0ba0 2023-11-24T06:25:54.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.802+0000 7eff667fc700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 0x7eff5406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:54.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.802+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff60088990 con 0x7eff700b0ba0 2023-11-24T06:25:54.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.802+0000 7eff754b1700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 0x7eff5406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:54.807 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.803+0000 7eff754b1700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 0x7eff5406d540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7eff70153ce0 tx=0x7eff6c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:54.809 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.807+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff600537c0 con 0x7eff700b0ba0 2023-11-24T06:25:54.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:54.992+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff700b1d70 con 0x7eff5406b080 2023-11-24T06:25:55.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.000+0000 7eff667fc700 1 -- 172.21.15.114:0/1059486943 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283479 (secure 0 0 0) 0x7eff700b1d70 con 0x7eff5406b080 2023-11-24T06:25:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:55.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 msgr2=0x7eff5406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 0x7eff5406d540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7eff70153ce0 tx=0x7eff6c01c040 comp rx=0 tx=0).stop 2023-11-24T06:25:55.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 msgr2=0x7eff7014e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7eff6000c4a0 tx=0x7eff6000c860 comp rx=0 tx=0).stop 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 shutdown_connections 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7eff5406b080 0x7eff5406d540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff700cd770 0x7eff7009c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff700b0ba0 0x7eff7014e480 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 --2- 172.21.15.114:0/1059486943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff700b0f80 0x7eff7009c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 >> 172.21.15.114:0/1059486943 conn(0x7eff700b57b0 msgr2=0x7eff700ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 shutdown_connections 2023-11-24T06:25:55.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.003+0000 7eff77f16700 1 -- 172.21.15.114:0/1059486943 wait complete. 2023-11-24T06:25:55.013 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:55.018 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-24T06:25:53.725877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221188,"kb_used_data":5012,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753532,"statfs":{"total":773094113280,"available":772867616768,"internally_reserved":0,"allocated":5132288,"data_stored":2747843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220180"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515714+0000","last_change":"2023-11-24T06:25:49.515714+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515714+0000","last_undegraded":"2023-11-24T06:25:49.515714+0000","last_fullsized":"2023-11-24T06:25:49.515714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522707+0000","last_change":"2023-11-24T06:25:49.522707+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522707+0000","last_undegraded":"2023-11-24T06:25:49.522707+0000","last_fullsized":"2023-11-24T06:25:49.522707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513933+0000","last_change":"2023-11-24T06:25:49.513933+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513933+0000","last_undegraded":"2023-11-24T06:25:49.513933+0000","last_fullsized":"2023-11-24T06:25:49.513933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521392+0000","last_change":"2023-11-24T06:25:49.521392+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521392+0000","last_undegraded":"2023-11-24T06:25:49.521392+0000","last_fullsized":"2023-11-24T06:25:49.521392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522718+0000","last_change":"2023-11-24T06:25:49.522718+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522718+0000","last_undegraded":"2023-11-24T06:25:49.522718+0000","last_fullsized":"2023-11-24T06:25:49.522718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526883+0000","last_change":"2023-11-24T06:25:49.526883+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526883+0000","last_undegraded":"2023-11-24T06:25:49.526883+0000","last_fullsized":"2023-11-24T06:25:49.526883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516210+0000","last_change":"2023-11-24T06:25:49.516210+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516210+0000","last_undegraded":"2023-11-24T06:25:49.516210+0000","last_fullsized":"2023-11-24T06:25:49.516210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521856+0000","last_change":"2023-11-24T06:25:49.521856+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521856+0000","last_undegraded":"2023-11-24T06:25:49.521856+0000","last_fullsized":"2023-11-24T06:25:49.521856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520072+0000","last_change":"2023-11-24T06:25:49.520072+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520072+0000","last_undegraded":"2023-11-24T06:25:49.520072+0000","last_fullsized":"2023-11-24T06:25:49.520072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520473+0000","last_change":"2023-11-24T06:25:49.520473+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520473+0000","last_undegraded":"2023-11-24T06:25:49.520473+0000","last_fullsized":"2023-11-24T06:25:49.520473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519995+0000","last_change":"2023-11-24T06:25:49.519995+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519995+0000","last_undegraded":"2023-11-24T06:25:49.519995+0000","last_fullsized":"2023-11-24T06:25:49.519995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524278+0000","last_change":"2023-11-24T06:25:49.524278+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524278+0000","last_undegraded":"2023-11-24T06:25:49.524278+0000","last_fullsized":"2023-11-24T06:25:49.524278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517000+0000","last_change":"2023-11-24T06:25:49.517000+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517000+0000","last_undegraded":"2023-11-24T06:25:49.517000+0000","last_fullsized":"2023-11-24T06:25:49.517000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520005+0000","last_change":"2023-11-24T06:25:49.520005+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520005+0000","last_undegraded":"2023-11-24T06:25:49.520005+0000","last_fullsized":"2023-11-24T06:25:49.520005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525682+0000","last_change":"2023-11-24T06:25:49.525682+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525682+0000","last_undegraded":"2023-11-24T06:25:49.525682+0000","last_fullsized":"2023-11-24T06:25:49.525682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517415+0000","last_change":"2023-11-24T06:25:49.517415+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517415+0000","last_undegraded":"2023-11-24T06:25:49.517415+0000","last_fullsized":"2023-11-24T06:25:49.517415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513815+0000","last_change":"2023-11-24T06:25:49.513815+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513815+0000","last_undegraded":"2023-11-24T06:25:49.513815+0000","last_fullsized":"2023-11-24T06:25:49.513815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515328+0000","last_change":"2023-11-24T06:25:49.515328+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515328+0000","last_undegraded":"2023-11-24T06:25:49.515328+0000","last_fullsized":"2023-11-24T06:25:49.515328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518515+0000","last_change":"2023-11-24T06:25:49.518515+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518515+0000","last_undegraded":"2023-11-24T06:25:49.518515+0000","last_fullsized":"2023-11-24T06:25:49.518515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516635+0000","last_change":"2023-11-24T06:25:49.516635+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516635+0000","last_undegraded":"2023-11-24T06:25:49.516635+0000","last_fullsized":"2023-11-24T06:25:49.516635+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.527437+0000","last_change":"2023-11-24T06:25:49.527437+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.527437+0000","last_undegraded":"2023-11-24T06:25:49.527437+0000","last_fullsized":"2023-11-24T06:25:49.527437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519517+0000","last_change":"2023-11-24T06:25:49.519517+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519517+0000","last_undegraded":"2023-11-24T06:25:49.519517+0000","last_fullsized":"2023-11-24T06:25:49.519517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518630+0000","last_change":"2023-11-24T06:25:49.518630+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518630+0000","last_undegraded":"2023-11-24T06:25:49.518630+0000","last_fullsized":"2023-11-24T06:25:49.518630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516923+0000","last_change":"2023-11-24T06:25:49.516923+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516923+0000","last_undegraded":"2023-11-24T06:25:49.516923+0000","last_fullsized":"2023-11-24T06:25:49.516923+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517579+0000","last_change":"2023-11-24T06:25:49.517579+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517579+0000","last_undegraded":"2023-11-24T06:25:49.517579+0000","last_fullsized":"2023-11-24T06:25:49.517579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516556+0000","last_change":"2023-11-24T06:25:49.516556+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516556+0000","last_undegraded":"2023-11-24T06:25:49.516556+0000","last_fullsized":"2023-11-24T06:25:49.516556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513365+0000","last_change":"2023-11-24T06:25:49.513365+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513365+0000","last_undegraded":"2023-11-24T06:25:49.513365+0000","last_fullsized":"2023-11-24T06:25:49.513365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525130+0000","last_change":"2023-11-24T06:25:49.525130+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525130+0000","last_undegraded":"2023-11-24T06:25:49.525130+0000","last_fullsized":"2023-11-24T06:25:49.525130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516966+0000","last_change":"2023-11-24T06:25:49.516966+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516966+0000","last_undegraded":"2023-11-24T06:25:49.516966+0000","last_fullsized":"2023-11-24T06:25:49.516966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.514385+0000","last_change":"2023-11-24T06:25:49.514385+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.514385+0000","last_undegraded":"2023-11-24T06:25:49.514385+0000","last_fullsized":"2023-11-24T06:25:49.514385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525215+0000","last_change":"2023-11-24T06:25:49.525215+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525215+0000","last_undegraded":"2023-11-24T06:25:49.525215+0000","last_fullsized":"2023-11-24T06:25:49.525215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516434+0000","last_change":"2023-11-24T06:25:49.516434+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516434+0000","last_undegraded":"2023-11-24T06:25:49.516434+0000","last_fullsized":"2023-11-24T06:25:49.516434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519016+0000","last_change":"2023-11-24T06:25:49.519016+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519016+0000","last_undegraded":"2023-11-24T06:25:49.519016+0000","last_fullsized":"2023-11-24T06:25:49.519016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534732+0000","last_change":"2023-11-24T06:25:49.531680+0000","last_active":"2023-11-24T06:25:49.534732+0000","last_peered":"2023-11-24T06:25:49.534732+0000","last_clean":"2023-11-24T06:25:49.534732+0000","last_became_active":"2023-11-24T06:25:49.531361+0000","last_became_peered":"2023-11-24T06:25:49.531361+0000","last_unstale":"2023-11-24T06:25:49.534732+0000","last_undegraded":"2023-11-24T06:25:49.534732+0000","last_fullsized":"2023-11-24T06:25:49.534732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519849+0000","last_change":"2023-11-24T06:25:49.519849+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519849+0000","last_undegraded":"2023-11-24T06:25:49.519849+0000","last_fullsized":"2023-11-24T06:25:49.519849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528570+0000","last_change":"2023-11-24T06:25:49.525389+0000","last_active":"2023-11-24T06:25:49.528570+0000","last_peered":"2023-11-24T06:25:49.528570+0000","last_clean":"2023-11-24T06:25:49.528570+0000","last_became_active":"2023-11-24T06:25:49.524856+0000","last_became_peered":"2023-11-24T06:25:49.524856+0000","last_unstale":"2023-11-24T06:25:49.528570+0000","last_undegraded":"2023-11-24T06:25:49.528570+0000","last_fullsized":"2023-11-24T06:25:49.528570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:11.371497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524684+0000","last_change":"2023-11-24T06:25:49.524684+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524684+0000","last_undegraded":"2023-11-24T06:25:49.524684+0000","last_fullsized":"2023-11-24T06:25:49.524684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534919+0000","last_change":"2023-11-24T06:25:49.530053+0000","last_active":"2023-11-24T06:25:49.534919+0000","last_peered":"2023-11-24T06:25:49.534919+0000","last_clean":"2023-11-24T06:25:49.534919+0000","last_became_active":"2023-11-24T06:25:49.529544+0000","last_became_peered":"2023-11-24T06:25:49.529544+0000","last_unstale":"2023-11-24T06:25:49.534919+0000","last_undegraded":"2023-11-24T06:25:49.534919+0000","last_fullsized":"2023-11-24T06:25:49.534919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:21.180992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518734+0000","last_change":"2023-11-24T06:25:49.518734+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518734+0000","last_undegraded":"2023-11-24T06:25:49.518734+0000","last_fullsized":"2023-11-24T06:25:49.518734+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531177+0000","last_change":"2023-11-24T06:25:49.529413+0000","last_active":"2023-11-24T06:25:49.531177+0000","last_peered":"2023-11-24T06:25:49.531177+0000","last_clean":"2023-11-24T06:25:49.531177+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.531177+0000","last_undegraded":"2023-11-24T06:25:49.531177+0000","last_fullsized":"2023-11-24T06:25:49.531177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513919+0000","last_change":"2023-11-24T06:25:49.513919+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513919+0000","last_undegraded":"2023-11-24T06:25:49.513919+0000","last_fullsized":"2023-11-24T06:25:49.513919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537262+0000","last_change":"2023-11-24T06:25:49.532801+0000","last_active":"2023-11-24T06:25:49.537262+0000","last_peered":"2023-11-24T06:25:49.537262+0000","last_clean":"2023-11-24T06:25:49.537262+0000","last_became_active":"2023-11-24T06:25:49.531990+0000","last_became_peered":"2023-11-24T06:25:49.531990+0000","last_unstale":"2023-11-24T06:25:49.537262+0000","last_undegraded":"2023-11-24T06:25:49.537262+0000","last_fullsized":"2023-11-24T06:25:49.537262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:58.851781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520725+0000","last_change":"2023-11-24T06:25:49.520725+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520725+0000","last_undegraded":"2023-11-24T06:25:49.520725+0000","last_fullsized":"2023-11-24T06:25:49.520725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535363+0000","last_change":"2023-11-24T06:25:49.529122+0000","last_active":"2023-11-24T06:25:49.535363+0000","last_peered":"2023-11-24T06:25:49.535363+0000","last_clean":"2023-11-24T06:25:49.535363+0000","last_became_active":"2023-11-24T06:25:49.528854+0000","last_became_peered":"2023-11-24T06:25:49.528854+0000","last_unstale":"2023-11-24T06:25:49.535363+0000","last_undegraded":"2023-11-24T06:25:49.535363+0000","last_fullsized":"2023-11-24T06:25:49.535363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:41.758296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519532+0000","last_change":"2023-11-24T06:25:49.519532+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519532+0000","last_undegraded":"2023-11-24T06:25:49.519532+0000","last_fullsized":"2023-11-24T06:25:49.519532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535377+0000","last_change":"2023-11-24T06:25:49.529495+0000","last_active":"2023-11-24T06:25:49.535377+0000","last_peered":"2023-11-24T06:25:49.535377+0000","last_clean":"2023-11-24T06:25:49.535377+0000","last_became_active":"2023-11-24T06:25:49.529256+0000","last_became_peered":"2023-11-24T06:25:49.529256+0000","last_unstale":"2023-11-24T06:25:49.535377+0000","last_undegraded":"2023-11-24T06:25:49.535377+0000","last_fullsized":"2023-11-24T06:25:49.535377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517845+0000","last_change":"2023-11-24T06:25:49.517845+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517845+0000","last_undegraded":"2023-11-24T06:25:49.517845+0000","last_fullsized":"2023-11-24T06:25:49.517845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.527150+0000","last_change":"2023-11-24T06:25:49.522891+0000","last_active":"2023-11-24T06:25:49.527150+0000","last_peered":"2023-11-24T06:25:49.527150+0000","last_clean":"2023-11-24T06:25:49.527150+0000","last_became_active":"2023-11-24T06:25:49.522605+0000","last_became_peered":"2023-11-24T06:25:49.522605+0000","last_unstale":"2023-11-24T06:25:49.527150+0000","last_undegraded":"2023-11-24T06:25:49.527150+0000","last_fullsized":"2023-11-24T06:25:49.527150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:54.895968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513631+0000","last_change":"2023-11-24T06:25:49.513631+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513631+0000","last_undegraded":"2023-11-24T06:25:49.513631+0000","last_fullsized":"2023-11-24T06:25:49.513631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525820+0000","last_change":"2023-11-24T06:25:49.522358+0000","last_active":"2023-11-24T06:25:49.525820+0000","last_peered":"2023-11-24T06:25:49.525820+0000","last_clean":"2023-11-24T06:25:49.525820+0000","last_became_active":"2023-11-24T06:25:49.521982+0000","last_became_peered":"2023-11-24T06:25:49.521982+0000","last_unstale":"2023-11-24T06:25:49.525820+0000","last_undegraded":"2023-11-24T06:25:49.525820+0000","last_fullsized":"2023-11-24T06:25:49.525820+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:36.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515185+0000","last_change":"2023-11-24T06:25:49.515185+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515185+0000","last_undegraded":"2023-11-24T06:25:49.515185+0000","last_fullsized":"2023-11-24T06:25:49.515185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528519+0000","last_change":"2023-11-24T06:25:49.524636+0000","last_active":"2023-11-24T06:25:49.528519+0000","last_peered":"2023-11-24T06:25:49.528519+0000","last_clean":"2023-11-24T06:25:49.528519+0000","last_became_active":"2023-11-24T06:25:49.524351+0000","last_became_peered":"2023-11-24T06:25:49.524351+0000","last_unstale":"2023-11-24T06:25:49.528519+0000","last_undegraded":"2023-11-24T06:25:49.528519+0000","last_fullsized":"2023-11-24T06:25:49.528519+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:32.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525547+0000","last_change":"2023-11-24T06:25:49.525547+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525547+0000","last_undegraded":"2023-11-24T06:25:49.525547+0000","last_fullsized":"2023-11-24T06:25:49.525547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.530358+0000","last_change":"2023-11-24T06:25:49.525281+0000","last_active":"2023-11-24T06:25:49.530358+0000","last_peered":"2023-11-24T06:25:49.530358+0000","last_clean":"2023-11-24T06:25:49.530358+0000","last_became_active":"2023-11-24T06:25:49.524899+0000","last_became_peered":"2023-11-24T06:25:49.524899+0000","last_unstale":"2023-11-24T06:25:49.530358+0000","last_undegraded":"2023-11-24T06:25:49.530358+0000","last_fullsized":"2023-11-24T06:25:49.530358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515945+0000","last_change":"2023-11-24T06:25:49.515945+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515945+0000","last_undegraded":"2023-11-24T06:25:49.515945+0000","last_fullsized":"2023-11-24T06:25:49.515945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.523174+0000","last_change":"2023-11-24T06:25:49.521264+0000","last_active":"2023-11-24T06:25:49.523174+0000","last_peered":"2023-11-24T06:25:49.523174+0000","last_clean":"2023-11-24T06:25:49.523174+0000","last_became_active":"2023-11-24T06:25:49.520985+0000","last_became_peered":"2023-11-24T06:25:49.520985+0000","last_unstale":"2023-11-24T06:25:49.523174+0000","last_undegraded":"2023-11-24T06:25:49.523174+0000","last_fullsized":"2023-11-24T06:25:49.523174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:51:01.319101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517033+0000","last_change":"2023-11-24T06:25:49.517033+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517033+0000","last_undegraded":"2023-11-24T06:25:49.517033+0000","last_fullsized":"2023-11-24T06:25:49.517033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532909+0000","last_change":"2023-11-24T06:25:49.531422+0000","last_active":"2023-11-24T06:25:49.532909+0000","last_peered":"2023-11-24T06:25:49.532909+0000","last_clean":"2023-11-24T06:25:49.532909+0000","last_became_active":"2023-11-24T06:25:49.531079+0000","last_became_peered":"2023-11-24T06:25:49.531079+0000","last_unstale":"2023-11-24T06:25:49.532909+0000","last_undegraded":"2023-11-24T06:25:49.532909+0000","last_fullsized":"2023-11-24T06:25:49.532909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:50.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520241+0000","last_change":"2023-11-24T06:25:49.520241+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520241+0000","last_undegraded":"2023-11-24T06:25:49.520241+0000","last_fullsized":"2023-11-24T06:25:49.520241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535343+0000","last_change":"2023-11-24T06:25:49.531424+0000","last_active":"2023-11-24T06:25:49.535343+0000","last_peered":"2023-11-24T06:25:49.535343+0000","last_clean":"2023-11-24T06:25:49.535343+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535343+0000","last_undegraded":"2023-11-24T06:25:49.535343+0000","last_fullsized":"2023-11-24T06:25:49.535343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524855+0000","last_change":"2023-11-24T06:25:49.524855+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524855+0000","last_undegraded":"2023-11-24T06:25:49.524855+0000","last_fullsized":"2023-11-24T06:25:49.524855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535002+0000","last_change":"2023-11-24T06:25:49.531559+0000","last_active":"2023-11-24T06:25:49.535002+0000","last_peered":"2023-11-24T06:25:49.535002+0000","last_clean":"2023-11-24T06:25:49.535002+0000","last_became_active":"2023-11-24T06:25:49.531127+0000","last_became_peered":"2023-11-24T06:25:49.531127+0000","last_unstale":"2023-11-24T06:25:49.535002+0000","last_undegraded":"2023-11-24T06:25:49.535002+0000","last_fullsized":"2023-11-24T06:25:49.535002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:05.025358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515749+0000","last_change":"2023-11-24T06:25:49.515749+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515749+0000","last_undegraded":"2023-11-24T06:25:49.515749+0000","last_fullsized":"2023-11-24T06:25:49.515749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.533260+0000","last_change":"2023-11-24T06:25:49.529384+0000","last_active":"2023-11-24T06:25:49.533260+0000","last_peered":"2023-11-24T06:25:49.533260+0000","last_clean":"2023-11-24T06:25:49.533260+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.533260+0000","last_undegraded":"2023-11-24T06:25:49.533260+0000","last_fullsized":"2023-11-24T06:25:49.533260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515868+0000","last_change":"2023-11-24T06:25:49.515868+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515868+0000","last_undegraded":"2023-11-24T06:25:49.515868+0000","last_fullsized":"2023-11-24T06:25:49.515868+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.538150+0000","last_change":"2023-11-24T06:25:49.533140+0000","last_active":"2023-11-24T06:25:49.538150+0000","last_peered":"2023-11-24T06:25:49.538150+0000","last_clean":"2023-11-24T06:25:49.538150+0000","last_became_active":"2023-11-24T06:25:49.532965+0000","last_became_peered":"2023-11-24T06:25:49.532965+0000","last_unstale":"2023-11-24T06:25:49.538150+0000","last_undegraded":"2023-11-24T06:25:49.538150+0000","last_fullsized":"2023-11-24T06:25:49.538150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:45.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518386+0000","last_change":"2023-11-24T06:25:49.518386+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518386+0000","last_undegraded":"2023-11-24T06:25:49.518386+0000","last_fullsized":"2023-11-24T06:25:49.518386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525822+0000","last_change":"2023-11-24T06:25:49.522374+0000","last_active":"2023-11-24T06:25:49.525822+0000","last_peered":"2023-11-24T06:25:49.525822+0000","last_clean":"2023-11-24T06:25:49.525822+0000","last_became_active":"2023-11-24T06:25:49.522053+0000","last_became_peered":"2023-11-24T06:25:49.522053+0000","last_unstale":"2023-11-24T06:25:49.525822+0000","last_undegraded":"2023-11-24T06:25:49.525822+0000","last_fullsized":"2023-11-24T06:25:49.525822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:55.240572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517595+0000","last_change":"2023-11-24T06:25:49.517595+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517595+0000","last_undegraded":"2023-11-24T06:25:49.517595+0000","last_fullsized":"2023-11-24T06:25:49.517595+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535288+0000","last_change":"2023-11-24T06:25:49.528898+0000","last_active":"2023-11-24T06:25:49.535288+0000","last_peered":"2023-11-24T06:25:49.535288+0000","last_clean":"2023-11-24T06:25:49.535288+0000","last_became_active":"2023-11-24T06:25:49.528642+0000","last_became_peered":"2023-11-24T06:25:49.528642+0000","last_unstale":"2023-11-24T06:25:49.535288+0000","last_undegraded":"2023-11-24T06:25:49.535288+0000","last_fullsized":"2023-11-24T06:25:49.535288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518084+0000","last_change":"2023-11-24T06:25:49.518084+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518084+0000","last_undegraded":"2023-11-24T06:25:49.518084+0000","last_fullsized":"2023-11-24T06:25:49.518084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534539+0000","last_change":"2023-11-24T06:25:49.531568+0000","last_active":"2023-11-24T06:25:49.534539+0000","last_peered":"2023-11-24T06:25:49.534539+0000","last_clean":"2023-11-24T06:25:49.534539+0000","last_became_active":"2023-11-24T06:25:49.531275+0000","last_became_peered":"2023-11-24T06:25:49.531275+0000","last_unstale":"2023-11-24T06:25:49.534539+0000","last_undegraded":"2023-11-24T06:25:49.534539+0000","last_fullsized":"2023-11-24T06:25:49.534539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525610+0000","last_change":"2023-11-24T06:25:49.525610+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525610+0000","last_undegraded":"2023-11-24T06:25:49.525610+0000","last_fullsized":"2023-11-24T06:25:49.525610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535112+0000","last_change":"2023-11-24T06:25:49.531979+0000","last_active":"2023-11-24T06:25:49.535112+0000","last_peered":"2023-11-24T06:25:49.535112+0000","last_clean":"2023-11-24T06:25:49.535112+0000","last_became_active":"2023-11-24T06:25:49.531699+0000","last_became_peered":"2023-11-24T06:25:49.531699+0000","last_unstale":"2023-11-24T06:25:49.535112+0000","last_undegraded":"2023-11-24T06:25:49.535112+0000","last_fullsized":"2023-11-24T06:25:49.535112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518503+0000","last_change":"2023-11-24T06:25:49.518503+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518503+0000","last_undegraded":"2023-11-24T06:25:49.518503+0000","last_fullsized":"2023-11-24T06:25:49.518503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535234+0000","last_change":"2023-11-24T06:25:49.528142+0000","last_active":"2023-11-24T06:25:49.535234+0000","last_peered":"2023-11-24T06:25:49.535234+0000","last_clean":"2023-11-24T06:25:49.535234+0000","last_became_active":"2023-11-24T06:25:49.527836+0000","last_became_peered":"2023-11-24T06:25:49.527836+0000","last_unstale":"2023-11-24T06:25:49.535234+0000","last_undegraded":"2023-11-24T06:25:49.535234+0000","last_fullsized":"2023-11-24T06:25:49.535234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515808+0000","last_change":"2023-11-24T06:25:49.515808+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515808+0000","last_undegraded":"2023-11-24T06:25:49.515808+0000","last_fullsized":"2023-11-24T06:25:49.515808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528512+0000","last_change":"2023-11-24T06:25:49.524448+0000","last_active":"2023-11-24T06:25:49.528512+0000","last_peered":"2023-11-24T06:25:49.528512+0000","last_clean":"2023-11-24T06:25:49.528512+0000","last_became_active":"2023-11-24T06:25:49.524159+0000","last_became_peered":"2023-11-24T06:25:49.524159+0000","last_unstale":"2023-11-24T06:25:49.528512+0000","last_undegraded":"2023-11-24T06:25:49.528512+0000","last_fullsized":"2023-11-24T06:25:49.528512+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:42.324117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518778+0000","last_change":"2023-11-24T06:25:49.518778+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518778+0000","last_undegraded":"2023-11-24T06:25:49.518778+0000","last_fullsized":"2023-11-24T06:25:49.518778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535306+0000","last_change":"2023-11-24T06:25:49.531285+0000","last_active":"2023-11-24T06:25:49.535306+0000","last_peered":"2023-11-24T06:25:49.535306+0000","last_clean":"2023-11-24T06:25:49.535306+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535306+0000","last_undegraded":"2023-11-24T06:25:49.535306+0000","last_fullsized":"2023-11-24T06:25:49.535306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518028+0000","last_change":"2023-11-24T06:25:49.518028+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518028+0000","last_undegraded":"2023-11-24T06:25:49.518028+0000","last_fullsized":"2023-11-24T06:25:49.518028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537271+0000","last_change":"2023-11-24T06:25:49.534058+0000","last_active":"2023-11-24T06:25:49.537271+0000","last_peered":"2023-11-24T06:25:49.537271+0000","last_clean":"2023-11-24T06:25:49.537271+0000","last_became_active":"2023-11-24T06:25:49.533728+0000","last_became_peered":"2023-11-24T06:25:49.533728+0000","last_unstale":"2023-11-24T06:25:49.537271+0000","last_undegraded":"2023-11-24T06:25:49.537271+0000","last_fullsized":"2023-11-24T06:25:49.537271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517885+0000","last_change":"2023-11-24T06:25:49.517885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517885+0000","last_undegraded":"2023-11-24T06:25:49.517885+0000","last_fullsized":"2023-11-24T06:25:49.517885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531262+0000","last_change":"2023-11-24T06:25:49.526982+0000","last_active":"2023-11-24T06:25:49.531262+0000","last_peered":"2023-11-24T06:25:49.531262+0000","last_clean":"2023-11-24T06:25:49.531262+0000","last_became_active":"2023-11-24T06:25:49.526688+0000","last_became_peered":"2023-11-24T06:25:49.526688+0000","last_unstale":"2023-11-24T06:25:49.531262+0000","last_undegraded":"2023-11-24T06:25:49.531262+0000","last_fullsized":"2023-11-24T06:25:49.531262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515368+0000","last_change":"2023-11-24T06:25:49.515368+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515368+0000","last_undegraded":"2023-11-24T06:25:49.515368+0000","last_fullsized":"2023-11-24T06:25:49.515368+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532904+0000","last_change":"2023-11-24T06:25:49.529663+0000","last_active":"2023-11-24T06:25:49.532904+0000","last_peered":"2023-11-24T06:25:49.532904+0000","last_clean":"2023-11-24T06:25:49.532904+0000","last_became_active":"2023-11-24T06:25:49.529401+0000","last_became_peered":"2023-11-24T06:25:49.529401+0000","last_unstale":"2023-11-24T06:25:49.532904+0000","last_undegraded":"2023-11-24T06:25:49.532904+0000","last_fullsized":"2023-11-24T06:25:49.532904+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:52.008057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524885+0000","last_change":"2023-11-24T06:25:49.524885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524885+0000","last_undegraded":"2023-11-24T06:25:49.524885+0000","last_fullsized":"2023-11-24T06:25:49.524885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531122+0000","last_change":"2023-11-24T06:25:49.525526+0000","last_active":"2023-11-24T06:25:49.531122+0000","last_peered":"2023-11-24T06:25:49.531122+0000","last_clean":"2023-11-24T06:25:49.531122+0000","last_became_active":"2023-11-24T06:25:49.525235+0000","last_became_peered":"2023-11-24T06:25:49.525235+0000","last_unstale":"2023-11-24T06:25:49.531122+0000","last_undegraded":"2023-11-24T06:25:49.531122+0000","last_fullsized":"2023-11-24T06:25:49.531122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526406+0000","last_change":"2023-11-24T06:25:49.526406+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526406+0000","last_undegraded":"2023-11-24T06:25:49.526406+0000","last_fullsized":"2023-11-24T06:25:49.526406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.522217+0000","last_change":"2023-11-24T06:25:49.520031+0000","last_active":"2023-11-24T06:25:49.522217+0000","last_peered":"2023-11-24T06:25:49.522217+0000","last_clean":"2023-11-24T06:25:49.522217+0000","last_became_active":"2023-11-24T06:25:49.519789+0000","last_became_peered":"2023-11-24T06:25:49.519789+0000","last_unstale":"2023-11-24T06:25:49.522217+0000","last_undegraded":"2023-11-24T06:25:49.522217+0000","last_fullsized":"2023-11-24T06:25:49.522217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.523607+0000","last_change":"2023-11-24T06:25:49.523607+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.523607+0000","last_undegraded":"2023-11-24T06:25:49.523607+0000","last_fullsized":"2023-11-24T06:25:49.523607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528237+0000","last_change":"2023-11-24T06:25:49.525157+0000","last_active":"2023-11-24T06:25:49.528237+0000","last_peered":"2023-11-24T06:25:49.528237+0000","last_clean":"2023-11-24T06:25:49.528237+0000","last_became_active":"2023-11-24T06:25:49.524896+0000","last_became_peered":"2023-11-24T06:25:49.524896+0000","last_unstale":"2023-11-24T06:25:49.528237+0000","last_undegraded":"2023-11-24T06:25:49.528237+0000","last_fullsized":"2023-11-24T06:25:49.528237+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513176+0000","last_change":"2023-11-24T06:25:49.513176+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513176+0000","last_undegraded":"2023-11-24T06:25:49.513176+0000","last_fullsized":"2023-11-24T06:25:49.513176+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.529913+0000","last_change":"2023-11-24T06:25:49.528916+0000","last_active":"2023-11-24T06:25:49.529913+0000","last_peered":"2023-11-24T06:25:49.529913+0000","last_clean":"2023-11-24T06:25:49.529913+0000","last_became_active":"2023-11-24T06:25:49.528145+0000","last_became_peered":"2023-11-24T06:25:49.528145+0000","last_unstale":"2023-11-24T06:25:49.529913+0000","last_undegraded":"2023-11-24T06:25:49.529913+0000","last_fullsized":"2023-11-24T06:25:49.529913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:18.021500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517915+0000","last_change":"2023-11-24T06:25:49.517915+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517915+0000","last_undegraded":"2023-11-24T06:25:49.517915+0000","last_fullsized":"2023-11-24T06:25:49.517915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537908+0000","last_change":"2023-11-24T06:25:49.533010+0000","last_active":"2023-11-24T06:25:49.537908+0000","last_peered":"2023-11-24T06:25:49.537908+0000","last_clean":"2023-11-24T06:25:49.537908+0000","last_became_active":"2023-11-24T06:25:49.532766+0000","last_became_peered":"2023-11-24T06:25:49.532766+0000","last_unstale":"2023-11-24T06:25:49.537908+0000","last_undegraded":"2023-11-24T06:25:49.537908+0000","last_fullsized":"2023-11-24T06:25:49.537908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-24T06:25:48.509286+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:48.509286+0000","last_peered":"2023-11-24T06:25:48.509286+0000","last_clean":"2023-11-24T06:25:48.509286+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:48.509286+0000","last_undegraded":"2023-11-24T06:25:48.509286+0000","last_fullsized":"2023-11-24T06:25:48.509286+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.495}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.439},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.747}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.315}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.371}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.747}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.379}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.752}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.89400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.499},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.51100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.441},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:55.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:55.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.177+0000 7f5bcf371700 1 Processor -- start 2023-11-24T06:25:55.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.177+0000 7f5bcf371700 1 -- start start 2023-11-24T06:25:55.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcf371700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815ddd0 0x7f5bc80b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcf371700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bc815e1b0 0x7f5bc80cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcf371700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc80b1d40 con 0x7f5bc80d1b30 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcf371700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80b28c0 con 0x7f5bc815e1b0 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcf371700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc806c000 con 0x7f5bc815ddd0 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bc815e1b0 0x7f5bc80cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcd90e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5bc80d1b30 0x7f5bc8161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56612/0 (socket says 172.21.15.114:56612) 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcd90e700 1 -- 172.21.15.114:0/1503050053 learned_addr learned my addr 172.21.15.114:0/1503050053 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcd10d700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815ddd0 0x7f5bc80b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 -- 172.21.15.114:0/1503050053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815ddd0 msgr2=0x7f5bc80b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815ddd0 0x7f5bc80b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 -- 172.21.15.114:0/1503050053 >> v1:172.21.15.114:6789/0 conn(0x7f5bc80d1b30 legacy=0x7f5bc8161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 -- 172.21.15.114:0/1503050053 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80b1f90 con 0x7f5bc815e1b0 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bbffff700 1 -- 172.21.15.114:0/1503050053 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643177371 0 0) 0x7f5bc80b1d40 con 0x7f5bc80d1b30 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc815e1b0 0x7f5bc80cf000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009840 tx=0x7f5bb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde45d56dfc9e3d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bbffff700 1 -- 172.21.15.114:0/1503050053 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8011830 con 0x7f5bc815e1b0 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bbffff700 1 -- 172.21.15.114:0/1503050053 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8011990 con 0x7f5bc815e1b0 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.178+0000 7f5bbffff700 1 -- 172.21.15.114:0/1503050053 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8011d20 con 0x7f5bc815e1b0 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 -- 172.21.15.114:0/1503050053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc815e1b0 msgr2=0x7f5bc80cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc815e1b0 0x7f5bc80cf000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009840 tx=0x7f5bb8009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 -- 172.21.15.114:0/1503050053 shutdown_connections 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815ddd0 0x7f5bc80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 --2- 172.21.15.114:0/1503050053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc815e1b0 0x7f5bc80cf000 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 -- 172.21.15.114:0/1503050053 >> 172.21.15.114:0/1503050053 conn(0x7f5bc80731a0 msgr2=0x7f5bc809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 -- 172.21.15.114:0/1503050053 shutdown_connections 2023-11-24T06:25:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.179+0000 7f5bcf371700 1 -- 172.21.15.114:0/1503050053 wait complete. 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.180+0000 7f5bcf371700 1 Processor -- start 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.180+0000 7f5bcf371700 1 -- start start 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 0x7f5bc809c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 0x7f5bc809f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc809dc40 con 0x7f5bc80d1b30 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc81519b0 con 0x7f5bc815ddd0 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcf371700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc809e5b0 con 0x7f5bc815e1b0 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40906/0 (socket says 172.21.15.114:40906) 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 -- 172.21.15.114:0/3805080899 learned_addr learned my addr 172.21.15.114:0/3805080899 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 -- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 msgr2=0x7f5bc809f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd90e700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 0x7f5bc809f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 0x7f5bc809c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 0x7f5bc809f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 -- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 msgr2=0x7f5bc809c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 0x7f5bc809c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 -- 172.21.15.114:0/3805080899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80094a0 con 0x7f5bc80d1b30 2023-11-24T06:25:55.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd90e700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 0x7f5bc809f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 0x7f5bc809c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.181+0000 7f5bcd10d700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400e810 tx=0x7f5bc400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.182+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4013610 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.182+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4013770 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.182+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc80b2ed0 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.182+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4013b20 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.182+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc809eb70 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.183+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc809f1c0 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.183+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5bc4013c80 con 0x7f5bc80d1b30 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.183+0000 7f5bbdffb700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 0x7f5bac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.184+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc4089750 con 0x7f5bc80d1b30 2023-11-24T06:25:55.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.184+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 0x7f5bac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.184+0000 7f5bcc90c700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 0x7f5bac06d540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815ac80 tx=0x7f5bb8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.188+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bc4054480 con 0x7f5bc80d1b30 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.373+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bc809e720 con 0x7f5bac06b080 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.381+0000 7f5bbdffb700 1 -- 172.21.15.114:0/3805080899 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283479 (secure 0 0 0) 0x7f5bc809e720 con 0x7f5bac06b080 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 msgr2=0x7f5bac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 0x7f5bac06d540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815ac80 tx=0x7f5bb8008030 comp rx=0 tx=0).stop 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 msgr2=0x7f5bc8151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400e810 tx=0x7f5bc400ebd0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 shutdown_connections 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5bac06b080 0x7f5bac06d540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc815e1b0 0x7f5bc809f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bc80d1b30 0x7f5bc8151470 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 --2- 172.21.15.114:0/3805080899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc815ddd0 0x7f5bc809c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 >> 172.21.15.114:0/3805080899 conn(0x7f5bc80731a0 msgr2=0x7f5bc80af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 shutdown_connections 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.384+0000 7f5bcf371700 1 -- 172.21.15.114:0/3805080899 wait complete. 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-24T06:25:53.725877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221188,"kb_used_data":5012,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753532,"statfs":{"total":773094113280,"available":772867616768,"internally_reserved":0,"allocated":5132288,"data_stored":2747843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220180"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515714+0000","last_change":"2023-11-24T06:25:49.515714+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515714+0000","last_undegraded":"2023-11-24T06:25:49.515714+0000","last_fullsized":"2023-11-24T06:25:49.515714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522707+0000","last_change":"2023-11-24T06:25:49.522707+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522707+0000","last_undegraded":"2023-11-24T06:25:49.522707+0000","last_fullsized":"2023-11-24T06:25:49.522707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513933+0000","last_change":"2023-11-24T06:25:49.513933+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513933+0000","last_undegraded":"2023-11-24T06:25:49.513933+0000","last_fullsized":"2023-11-24T06:25:49.513933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521392+0000","last_change":"2023-11-24T06:25:49.521392+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521392+0000","last_undegraded":"2023-11-24T06:25:49.521392+0000","last_fullsized":"2023-11-24T06:25:49.521392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.522718+0000","last_change":"2023-11-24T06:25:49.522718+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.522718+0000","last_undegraded":"2023-11-24T06:25:49.522718+0000","last_fullsized":"2023-11-24T06:25:49.522718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526883+0000","last_change":"2023-11-24T06:25:49.526883+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526883+0000","last_undegraded":"2023-11-24T06:25:49.526883+0000","last_fullsized":"2023-11-24T06:25:49.526883+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516210+0000","last_change":"2023-11-24T06:25:49.516210+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516210+0000","last_undegraded":"2023-11-24T06:25:49.516210+0000","last_fullsized":"2023-11-24T06:25:49.516210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.521856+0000","last_change":"2023-11-24T06:25:49.521856+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.521856+0000","last_undegraded":"2023-11-24T06:25:49.521856+0000","last_fullsized":"2023-11-24T06:25:49.521856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520072+0000","last_change":"2023-11-24T06:25:49.520072+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520072+0000","last_undegraded":"2023-11-24T06:25:49.520072+0000","last_fullsized":"2023-11-24T06:25:49.520072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520473+0000","last_change":"2023-11-24T06:25:49.520473+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520473+0000","last_undegraded":"2023-11-24T06:25:49.520473+0000","last_fullsized":"2023-11-24T06:25:49.520473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519995+0000","last_change":"2023-11-24T06:25:49.519995+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519995+0000","last_undegraded":"2023-11-24T06:25:49.519995+0000","last_fullsized":"2023-11-24T06:25:49.519995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524278+0000","last_change":"2023-11-24T06:25:49.524278+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524278+0000","last_undegraded":"2023-11-24T06:25:49.524278+0000","last_fullsized":"2023-11-24T06:25:49.524278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517000+0000","last_change":"2023-11-24T06:25:49.517000+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517000+0000","last_undegraded":"2023-11-24T06:25:49.517000+0000","last_fullsized":"2023-11-24T06:25:49.517000+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520005+0000","last_change":"2023-11-24T06:25:49.520005+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520005+0000","last_undegraded":"2023-11-24T06:25:49.520005+0000","last_fullsized":"2023-11-24T06:25:49.520005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525682+0000","last_change":"2023-11-24T06:25:49.525682+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525682+0000","last_undegraded":"2023-11-24T06:25:49.525682+0000","last_fullsized":"2023-11-24T06:25:49.525682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517415+0000","last_change":"2023-11-24T06:25:49.517415+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517415+0000","last_undegraded":"2023-11-24T06:25:49.517415+0000","last_fullsized":"2023-11-24T06:25:49.517415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513815+0000","last_change":"2023-11-24T06:25:49.513815+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513815+0000","last_undegraded":"2023-11-24T06:25:49.513815+0000","last_fullsized":"2023-11-24T06:25:49.513815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515328+0000","last_change":"2023-11-24T06:25:49.515328+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515328+0000","last_undegraded":"2023-11-24T06:25:49.515328+0000","last_fullsized":"2023-11-24T06:25:49.515328+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518515+0000","last_change":"2023-11-24T06:25:49.518515+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518515+0000","last_undegraded":"2023-11-24T06:25:49.518515+0000","last_fullsized":"2023-11-24T06:25:49.518515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516635+0000","last_change":"2023-11-24T06:25:49.516635+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516635+0000","last_undegraded":"2023-11-24T06:25:49.516635+0000","last_fullsized":"2023-11-24T06:25:49.516635+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.527437+0000","last_change":"2023-11-24T06:25:49.527437+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.527437+0000","last_undegraded":"2023-11-24T06:25:49.527437+0000","last_fullsized":"2023-11-24T06:25:49.527437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519517+0000","last_change":"2023-11-24T06:25:49.519517+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519517+0000","last_undegraded":"2023-11-24T06:25:49.519517+0000","last_fullsized":"2023-11-24T06:25:49.519517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518630+0000","last_change":"2023-11-24T06:25:49.518630+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518630+0000","last_undegraded":"2023-11-24T06:25:49.518630+0000","last_fullsized":"2023-11-24T06:25:49.518630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516923+0000","last_change":"2023-11-24T06:25:49.516923+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516923+0000","last_undegraded":"2023-11-24T06:25:49.516923+0000","last_fullsized":"2023-11-24T06:25:49.516923+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517579+0000","last_change":"2023-11-24T06:25:49.517579+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517579+0000","last_undegraded":"2023-11-24T06:25:49.517579+0000","last_fullsized":"2023-11-24T06:25:49.517579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516556+0000","last_change":"2023-11-24T06:25:49.516556+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516556+0000","last_undegraded":"2023-11-24T06:25:49.516556+0000","last_fullsized":"2023-11-24T06:25:49.516556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513365+0000","last_change":"2023-11-24T06:25:49.513365+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513365+0000","last_undegraded":"2023-11-24T06:25:49.513365+0000","last_fullsized":"2023-11-24T06:25:49.513365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525130+0000","last_change":"2023-11-24T06:25:49.525130+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525130+0000","last_undegraded":"2023-11-24T06:25:49.525130+0000","last_fullsized":"2023-11-24T06:25:49.525130+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516966+0000","last_change":"2023-11-24T06:25:49.516966+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516966+0000","last_undegraded":"2023-11-24T06:25:49.516966+0000","last_fullsized":"2023-11-24T06:25:49.516966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.514385+0000","last_change":"2023-11-24T06:25:49.514385+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.514385+0000","last_undegraded":"2023-11-24T06:25:49.514385+0000","last_fullsized":"2023-11-24T06:25:49.514385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525215+0000","last_change":"2023-11-24T06:25:49.525215+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525215+0000","last_undegraded":"2023-11-24T06:25:49.525215+0000","last_fullsized":"2023-11-24T06:25:49.525215+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.516434+0000","last_change":"2023-11-24T06:25:49.516434+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.516434+0000","last_undegraded":"2023-11-24T06:25:49.516434+0000","last_fullsized":"2023-11-24T06:25:49.516434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519016+0000","last_change":"2023-11-24T06:25:49.519016+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519016+0000","last_undegraded":"2023-11-24T06:25:49.519016+0000","last_fullsized":"2023-11-24T06:25:49.519016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534732+0000","last_change":"2023-11-24T06:25:49.531680+0000","last_active":"2023-11-24T06:25:49.534732+0000","last_peered":"2023-11-24T06:25:49.534732+0000","last_clean":"2023-11-24T06:25:49.534732+0000","last_became_active":"2023-11-24T06:25:49.531361+0000","last_became_peered":"2023-11-24T06:25:49.531361+0000","last_unstale":"2023-11-24T06:25:49.534732+0000","last_undegraded":"2023-11-24T06:25:49.534732+0000","last_fullsized":"2023-11-24T06:25:49.534732+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519849+0000","last_change":"2023-11-24T06:25:49.519849+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519849+0000","last_undegraded":"2023-11-24T06:25:49.519849+0000","last_fullsized":"2023-11-24T06:25:49.519849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528570+0000","last_change":"2023-11-24T06:25:49.525389+0000","last_active":"2023-11-24T06:25:49.528570+0000","last_peered":"2023-11-24T06:25:49.528570+0000","last_clean":"2023-11-24T06:25:49.528570+0000","last_became_active":"2023-11-24T06:25:49.524856+0000","last_became_peered":"2023-11-24T06:25:49.524856+0000","last_unstale":"2023-11-24T06:25:49.528570+0000","last_undegraded":"2023-11-24T06:25:49.528570+0000","last_fullsized":"2023-11-24T06:25:49.528570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:11.371497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524684+0000","last_change":"2023-11-24T06:25:49.524684+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524684+0000","last_undegraded":"2023-11-24T06:25:49.524684+0000","last_fullsized":"2023-11-24T06:25:49.524684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534919+0000","last_change":"2023-11-24T06:25:49.530053+0000","last_active":"2023-11-24T06:25:49.534919+0000","last_peered":"2023-11-24T06:25:49.534919+0000","last_clean":"2023-11-24T06:25:49.534919+0000","last_became_active":"2023-11-24T06:25:49.529544+0000","last_became_peered":"2023-11-24T06:25:49.529544+0000","last_unstale":"2023-11-24T06:25:49.534919+0000","last_undegraded":"2023-11-24T06:25:49.534919+0000","last_fullsized":"2023-11-24T06:25:49.534919+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:21.180992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518734+0000","last_change":"2023-11-24T06:25:49.518734+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518734+0000","last_undegraded":"2023-11-24T06:25:49.518734+0000","last_fullsized":"2023-11-24T06:25:49.518734+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531177+0000","last_change":"2023-11-24T06:25:49.529413+0000","last_active":"2023-11-24T06:25:49.531177+0000","last_peered":"2023-11-24T06:25:49.531177+0000","last_clean":"2023-11-24T06:25:49.531177+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.531177+0000","last_undegraded":"2023-11-24T06:25:49.531177+0000","last_fullsized":"2023-11-24T06:25:49.531177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513919+0000","last_change":"2023-11-24T06:25:49.513919+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513919+0000","last_undegraded":"2023-11-24T06:25:49.513919+0000","last_fullsized":"2023-11-24T06:25:49.513919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537262+0000","last_change":"2023-11-24T06:25:49.532801+0000","last_active":"2023-11-24T06:25:49.537262+0000","last_peered":"2023-11-24T06:25:49.537262+0000","last_clean":"2023-11-24T06:25:49.537262+0000","last_became_active":"2023-11-24T06:25:49.531990+0000","last_became_peered":"2023-11-24T06:25:49.531990+0000","last_unstale":"2023-11-24T06:25:49.537262+0000","last_undegraded":"2023-11-24T06:25:49.537262+0000","last_fullsized":"2023-11-24T06:25:49.537262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:58.851781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520725+0000","last_change":"2023-11-24T06:25:49.520725+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520725+0000","last_undegraded":"2023-11-24T06:25:49.520725+0000","last_fullsized":"2023-11-24T06:25:49.520725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535363+0000","last_change":"2023-11-24T06:25:49.529122+0000","last_active":"2023-11-24T06:25:49.535363+0000","last_peered":"2023-11-24T06:25:49.535363+0000","last_clean":"2023-11-24T06:25:49.535363+0000","last_became_active":"2023-11-24T06:25:49.528854+0000","last_became_peered":"2023-11-24T06:25:49.528854+0000","last_unstale":"2023-11-24T06:25:49.535363+0000","last_undegraded":"2023-11-24T06:25:49.535363+0000","last_fullsized":"2023-11-24T06:25:49.535363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:41.758296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.519532+0000","last_change":"2023-11-24T06:25:49.519532+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.519532+0000","last_undegraded":"2023-11-24T06:25:49.519532+0000","last_fullsized":"2023-11-24T06:25:49.519532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535377+0000","last_change":"2023-11-24T06:25:49.529495+0000","last_active":"2023-11-24T06:25:49.535377+0000","last_peered":"2023-11-24T06:25:49.535377+0000","last_clean":"2023-11-24T06:25:49.535377+0000","last_became_active":"2023-11-24T06:25:49.529256+0000","last_became_peered":"2023-11-24T06:25:49.529256+0000","last_unstale":"2023-11-24T06:25:49.535377+0000","last_undegraded":"2023-11-24T06:25:49.535377+0000","last_fullsized":"2023-11-24T06:25:49.535377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517845+0000","last_change":"2023-11-24T06:25:49.517845+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517845+0000","last_undegraded":"2023-11-24T06:25:49.517845+0000","last_fullsized":"2023-11-24T06:25:49.517845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.527150+0000","last_change":"2023-11-24T06:25:49.522891+0000","last_active":"2023-11-24T06:25:49.527150+0000","last_peered":"2023-11-24T06:25:49.527150+0000","last_clean":"2023-11-24T06:25:49.527150+0000","last_became_active":"2023-11-24T06:25:49.522605+0000","last_became_peered":"2023-11-24T06:25:49.522605+0000","last_unstale":"2023-11-24T06:25:49.527150+0000","last_undegraded":"2023-11-24T06:25:49.527150+0000","last_fullsized":"2023-11-24T06:25:49.527150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:54.895968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513631+0000","last_change":"2023-11-24T06:25:49.513631+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513631+0000","last_undegraded":"2023-11-24T06:25:49.513631+0000","last_fullsized":"2023-11-24T06:25:49.513631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525820+0000","last_change":"2023-11-24T06:25:49.522358+0000","last_active":"2023-11-24T06:25:49.525820+0000","last_peered":"2023-11-24T06:25:49.525820+0000","last_clean":"2023-11-24T06:25:49.525820+0000","last_became_active":"2023-11-24T06:25:49.521982+0000","last_became_peered":"2023-11-24T06:25:49.521982+0000","last_unstale":"2023-11-24T06:25:49.525820+0000","last_undegraded":"2023-11-24T06:25:49.525820+0000","last_fullsized":"2023-11-24T06:25:49.525820+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:36.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515185+0000","last_change":"2023-11-24T06:25:49.515185+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515185+0000","last_undegraded":"2023-11-24T06:25:49.515185+0000","last_fullsized":"2023-11-24T06:25:49.515185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528519+0000","last_change":"2023-11-24T06:25:49.524636+0000","last_active":"2023-11-24T06:25:49.528519+0000","last_peered":"2023-11-24T06:25:49.528519+0000","last_clean":"2023-11-24T06:25:49.528519+0000","last_became_active":"2023-11-24T06:25:49.524351+0000","last_became_peered":"2023-11-24T06:25:49.524351+0000","last_unstale":"2023-11-24T06:25:49.528519+0000","last_undegraded":"2023-11-24T06:25:49.528519+0000","last_fullsized":"2023-11-24T06:25:49.528519+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:32.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525547+0000","last_change":"2023-11-24T06:25:49.525547+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525547+0000","last_undegraded":"2023-11-24T06:25:49.525547+0000","last_fullsized":"2023-11-24T06:25:49.525547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.530358+0000","last_change":"2023-11-24T06:25:49.525281+0000","last_active":"2023-11-24T06:25:49.530358+0000","last_peered":"2023-11-24T06:25:49.530358+0000","last_clean":"2023-11-24T06:25:49.530358+0000","last_became_active":"2023-11-24T06:25:49.524899+0000","last_became_peered":"2023-11-24T06:25:49.524899+0000","last_unstale":"2023-11-24T06:25:49.530358+0000","last_undegraded":"2023-11-24T06:25:49.530358+0000","last_fullsized":"2023-11-24T06:25:49.530358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515945+0000","last_change":"2023-11-24T06:25:49.515945+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515945+0000","last_undegraded":"2023-11-24T06:25:49.515945+0000","last_fullsized":"2023-11-24T06:25:49.515945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.523174+0000","last_change":"2023-11-24T06:25:49.521264+0000","last_active":"2023-11-24T06:25:49.523174+0000","last_peered":"2023-11-24T06:25:49.523174+0000","last_clean":"2023-11-24T06:25:49.523174+0000","last_became_active":"2023-11-24T06:25:49.520985+0000","last_became_peered":"2023-11-24T06:25:49.520985+0000","last_unstale":"2023-11-24T06:25:49.523174+0000","last_undegraded":"2023-11-24T06:25:49.523174+0000","last_fullsized":"2023-11-24T06:25:49.523174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:51:01.319101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517033+0000","last_change":"2023-11-24T06:25:49.517033+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517033+0000","last_undegraded":"2023-11-24T06:25:49.517033+0000","last_fullsized":"2023-11-24T06:25:49.517033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532909+0000","last_change":"2023-11-24T06:25:49.531422+0000","last_active":"2023-11-24T06:25:49.532909+0000","last_peered":"2023-11-24T06:25:49.532909+0000","last_clean":"2023-11-24T06:25:49.532909+0000","last_became_active":"2023-11-24T06:25:49.531079+0000","last_became_peered":"2023-11-24T06:25:49.531079+0000","last_unstale":"2023-11-24T06:25:49.532909+0000","last_undegraded":"2023-11-24T06:25:49.532909+0000","last_fullsized":"2023-11-24T06:25:49.532909+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:50.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.520241+0000","last_change":"2023-11-24T06:25:49.520241+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.520241+0000","last_undegraded":"2023-11-24T06:25:49.520241+0000","last_fullsized":"2023-11-24T06:25:49.520241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535343+0000","last_change":"2023-11-24T06:25:49.531424+0000","last_active":"2023-11-24T06:25:49.535343+0000","last_peered":"2023-11-24T06:25:49.535343+0000","last_clean":"2023-11-24T06:25:49.535343+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535343+0000","last_undegraded":"2023-11-24T06:25:49.535343+0000","last_fullsized":"2023-11-24T06:25:49.535343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:50.878146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524855+0000","last_change":"2023-11-24T06:25:49.524855+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524855+0000","last_undegraded":"2023-11-24T06:25:49.524855+0000","last_fullsized":"2023-11-24T06:25:49.524855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535002+0000","last_change":"2023-11-24T06:25:49.531559+0000","last_active":"2023-11-24T06:25:49.535002+0000","last_peered":"2023-11-24T06:25:49.535002+0000","last_clean":"2023-11-24T06:25:49.535002+0000","last_became_active":"2023-11-24T06:25:49.531127+0000","last_became_peered":"2023-11-24T06:25:49.531127+0000","last_unstale":"2023-11-24T06:25:49.535002+0000","last_undegraded":"2023-11-24T06:25:49.535002+0000","last_fullsized":"2023-11-24T06:25:49.535002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:05.025358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515749+0000","last_change":"2023-11-24T06:25:49.515749+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515749+0000","last_undegraded":"2023-11-24T06:25:49.515749+0000","last_fullsized":"2023-11-24T06:25:49.515749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.533260+0000","last_change":"2023-11-24T06:25:49.529384+0000","last_active":"2023-11-24T06:25:49.533260+0000","last_peered":"2023-11-24T06:25:49.533260+0000","last_clean":"2023-11-24T06:25:49.533260+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:49.533260+0000","last_undegraded":"2023-11-24T06:25:49.533260+0000","last_fullsized":"2023-11-24T06:25:49.533260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515868+0000","last_change":"2023-11-24T06:25:49.515868+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515868+0000","last_undegraded":"2023-11-24T06:25:49.515868+0000","last_fullsized":"2023-11-24T06:25:49.515868+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.538150+0000","last_change":"2023-11-24T06:25:49.533140+0000","last_active":"2023-11-24T06:25:49.538150+0000","last_peered":"2023-11-24T06:25:49.538150+0000","last_clean":"2023-11-24T06:25:49.538150+0000","last_became_active":"2023-11-24T06:25:49.532965+0000","last_became_peered":"2023-11-24T06:25:49.532965+0000","last_unstale":"2023-11-24T06:25:49.538150+0000","last_undegraded":"2023-11-24T06:25:49.538150+0000","last_fullsized":"2023-11-24T06:25:49.538150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:46:45.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518386+0000","last_change":"2023-11-24T06:25:49.518386+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518386+0000","last_undegraded":"2023-11-24T06:25:49.518386+0000","last_fullsized":"2023-11-24T06:25:49.518386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.525822+0000","last_change":"2023-11-24T06:25:49.522374+0000","last_active":"2023-11-24T06:25:49.525822+0000","last_peered":"2023-11-24T06:25:49.525822+0000","last_clean":"2023-11-24T06:25:49.525822+0000","last_became_active":"2023-11-24T06:25:49.522053+0000","last_became_peered":"2023-11-24T06:25:49.522053+0000","last_unstale":"2023-11-24T06:25:49.525822+0000","last_undegraded":"2023-11-24T06:25:49.525822+0000","last_fullsized":"2023-11-24T06:25:49.525822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:55.240572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517595+0000","last_change":"2023-11-24T06:25:49.517595+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517595+0000","last_undegraded":"2023-11-24T06:25:49.517595+0000","last_fullsized":"2023-11-24T06:25:49.517595+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535288+0000","last_change":"2023-11-24T06:25:49.528898+0000","last_active":"2023-11-24T06:25:49.535288+0000","last_peered":"2023-11-24T06:25:49.535288+0000","last_clean":"2023-11-24T06:25:49.535288+0000","last_became_active":"2023-11-24T06:25:49.528642+0000","last_became_peered":"2023-11-24T06:25:49.528642+0000","last_unstale":"2023-11-24T06:25:49.535288+0000","last_undegraded":"2023-11-24T06:25:49.535288+0000","last_fullsized":"2023-11-24T06:25:49.535288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:07.470140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518084+0000","last_change":"2023-11-24T06:25:49.518084+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518084+0000","last_undegraded":"2023-11-24T06:25:49.518084+0000","last_fullsized":"2023-11-24T06:25:49.518084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.534539+0000","last_change":"2023-11-24T06:25:49.531568+0000","last_active":"2023-11-24T06:25:49.534539+0000","last_peered":"2023-11-24T06:25:49.534539+0000","last_clean":"2023-11-24T06:25:49.534539+0000","last_became_active":"2023-11-24T06:25:49.531275+0000","last_became_peered":"2023-11-24T06:25:49.531275+0000","last_unstale":"2023-11-24T06:25:49.534539+0000","last_undegraded":"2023-11-24T06:25:49.534539+0000","last_fullsized":"2023-11-24T06:25:49.534539+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.525610+0000","last_change":"2023-11-24T06:25:49.525610+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.525610+0000","last_undegraded":"2023-11-24T06:25:49.525610+0000","last_fullsized":"2023-11-24T06:25:49.525610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535112+0000","last_change":"2023-11-24T06:25:49.531979+0000","last_active":"2023-11-24T06:25:49.535112+0000","last_peered":"2023-11-24T06:25:49.535112+0000","last_clean":"2023-11-24T06:25:49.535112+0000","last_became_active":"2023-11-24T06:25:49.531699+0000","last_became_peered":"2023-11-24T06:25:49.531699+0000","last_unstale":"2023-11-24T06:25:49.535112+0000","last_undegraded":"2023-11-24T06:25:49.535112+0000","last_fullsized":"2023-11-24T06:25:49.535112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518503+0000","last_change":"2023-11-24T06:25:49.518503+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518503+0000","last_undegraded":"2023-11-24T06:25:49.518503+0000","last_fullsized":"2023-11-24T06:25:49.518503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535234+0000","last_change":"2023-11-24T06:25:49.528142+0000","last_active":"2023-11-24T06:25:49.535234+0000","last_peered":"2023-11-24T06:25:49.535234+0000","last_clean":"2023-11-24T06:25:49.535234+0000","last_became_active":"2023-11-24T06:25:49.527836+0000","last_became_peered":"2023-11-24T06:25:49.527836+0000","last_unstale":"2023-11-24T06:25:49.535234+0000","last_undegraded":"2023-11-24T06:25:49.535234+0000","last_fullsized":"2023-11-24T06:25:49.535234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515808+0000","last_change":"2023-11-24T06:25:49.515808+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515808+0000","last_undegraded":"2023-11-24T06:25:49.515808+0000","last_fullsized":"2023-11-24T06:25:49.515808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528512+0000","last_change":"2023-11-24T06:25:49.524448+0000","last_active":"2023-11-24T06:25:49.528512+0000","last_peered":"2023-11-24T06:25:49.528512+0000","last_clean":"2023-11-24T06:25:49.528512+0000","last_became_active":"2023-11-24T06:25:49.524159+0000","last_became_peered":"2023-11-24T06:25:49.524159+0000","last_unstale":"2023-11-24T06:25:49.528512+0000","last_undegraded":"2023-11-24T06:25:49.528512+0000","last_fullsized":"2023-11-24T06:25:49.528512+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:42.324117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518778+0000","last_change":"2023-11-24T06:25:49.518778+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518778+0000","last_undegraded":"2023-11-24T06:25:49.518778+0000","last_fullsized":"2023-11-24T06:25:49.518778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.535306+0000","last_change":"2023-11-24T06:25:49.531285+0000","last_active":"2023-11-24T06:25:49.535306+0000","last_peered":"2023-11-24T06:25:49.535306+0000","last_clean":"2023-11-24T06:25:49.535306+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:49.535306+0000","last_undegraded":"2023-11-24T06:25:49.535306+0000","last_fullsized":"2023-11-24T06:25:49.535306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.518028+0000","last_change":"2023-11-24T06:25:49.518028+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.518028+0000","last_undegraded":"2023-11-24T06:25:49.518028+0000","last_fullsized":"2023-11-24T06:25:49.518028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537271+0000","last_change":"2023-11-24T06:25:49.534058+0000","last_active":"2023-11-24T06:25:49.537271+0000","last_peered":"2023-11-24T06:25:49.537271+0000","last_clean":"2023-11-24T06:25:49.537271+0000","last_became_active":"2023-11-24T06:25:49.533728+0000","last_became_peered":"2023-11-24T06:25:49.533728+0000","last_unstale":"2023-11-24T06:25:49.537271+0000","last_undegraded":"2023-11-24T06:25:49.537271+0000","last_fullsized":"2023-11-24T06:25:49.537271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517885+0000","last_change":"2023-11-24T06:25:49.517885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517885+0000","last_undegraded":"2023-11-24T06:25:49.517885+0000","last_fullsized":"2023-11-24T06:25:49.517885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531262+0000","last_change":"2023-11-24T06:25:49.526982+0000","last_active":"2023-11-24T06:25:49.531262+0000","last_peered":"2023-11-24T06:25:49.531262+0000","last_clean":"2023-11-24T06:25:49.531262+0000","last_became_active":"2023-11-24T06:25:49.526688+0000","last_became_peered":"2023-11-24T06:25:49.526688+0000","last_unstale":"2023-11-24T06:25:49.531262+0000","last_undegraded":"2023-11-24T06:25:49.531262+0000","last_fullsized":"2023-11-24T06:25:49.531262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:43.184228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.515368+0000","last_change":"2023-11-24T06:25:49.515368+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.515368+0000","last_undegraded":"2023-11-24T06:25:49.515368+0000","last_fullsized":"2023-11-24T06:25:49.515368+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.532904+0000","last_change":"2023-11-24T06:25:49.529663+0000","last_active":"2023-11-24T06:25:49.532904+0000","last_peered":"2023-11-24T06:25:49.532904+0000","last_clean":"2023-11-24T06:25:49.532904+0000","last_became_active":"2023-11-24T06:25:49.529401+0000","last_became_peered":"2023-11-24T06:25:49.529401+0000","last_unstale":"2023-11-24T06:25:49.532904+0000","last_undegraded":"2023-11-24T06:25:49.532904+0000","last_fullsized":"2023-11-24T06:25:49.532904+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:52.008057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.524885+0000","last_change":"2023-11-24T06:25:49.524885+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.524885+0000","last_undegraded":"2023-11-24T06:25:49.524885+0000","last_fullsized":"2023-11-24T06:25:49.524885+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.531122+0000","last_change":"2023-11-24T06:25:49.525526+0000","last_active":"2023-11-24T06:25:49.531122+0000","last_peered":"2023-11-24T06:25:49.531122+0000","last_clean":"2023-11-24T06:25:49.531122+0000","last_became_active":"2023-11-24T06:25:49.525235+0000","last_became_peered":"2023-11-24T06:25:49.525235+0000","last_unstale":"2023-11-24T06:25:49.531122+0000","last_undegraded":"2023-11-24T06:25:49.531122+0000","last_fullsized":"2023-11-24T06:25:49.531122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:57.725011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.526406+0000","last_change":"2023-11-24T06:25:49.526406+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.526406+0000","last_undegraded":"2023-11-24T06:25:49.526406+0000","last_fullsized":"2023-11-24T06:25:49.526406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.522217+0000","last_change":"2023-11-24T06:25:49.520031+0000","last_active":"2023-11-24T06:25:49.522217+0000","last_peered":"2023-11-24T06:25:49.522217+0000","last_clean":"2023-11-24T06:25:49.522217+0000","last_became_active":"2023-11-24T06:25:49.519789+0000","last_became_peered":"2023-11-24T06:25:49.519789+0000","last_unstale":"2023-11-24T06:25:49.522217+0000","last_undegraded":"2023-11-24T06:25:49.522217+0000","last_fullsized":"2023-11-24T06:25:49.522217+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:42.367515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.523607+0000","last_change":"2023-11-24T06:25:49.523607+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.523607+0000","last_undegraded":"2023-11-24T06:25:49.523607+0000","last_fullsized":"2023-11-24T06:25:49.523607+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.528237+0000","last_change":"2023-11-24T06:25:49.525157+0000","last_active":"2023-11-24T06:25:49.528237+0000","last_peered":"2023-11-24T06:25:49.528237+0000","last_clean":"2023-11-24T06:25:49.528237+0000","last_became_active":"2023-11-24T06:25:49.524896+0000","last_became_peered":"2023-11-24T06:25:49.524896+0000","last_unstale":"2023-11-24T06:25:49.528237+0000","last_undegraded":"2023-11-24T06:25:49.528237+0000","last_fullsized":"2023-11-24T06:25:49.528237+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.513176+0000","last_change":"2023-11-24T06:25:49.513176+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.513176+0000","last_undegraded":"2023-11-24T06:25:49.513176+0000","last_fullsized":"2023-11-24T06:25:49.513176+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.529913+0000","last_change":"2023-11-24T06:25:49.528916+0000","last_active":"2023-11-24T06:25:49.529913+0000","last_peered":"2023-11-24T06:25:49.529913+0000","last_clean":"2023-11-24T06:25:49.529913+0000","last_became_active":"2023-11-24T06:25:49.528145+0000","last_became_peered":"2023-11-24T06:25:49.528145+0000","last_unstale":"2023-11-24T06:25:49.529913+0000","last_undegraded":"2023-11-24T06:25:49.529913+0000","last_fullsized":"2023-11-24T06:25:49.529913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:18.021500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-24T06:25:49.517915+0000","last_change":"2023-11-24T06:25:49.517915+0000","last_active":"2023-11-24T06:25:49.505313+0000","last_peered":"2023-11-24T06:25:49.505313+0000","last_clean":"2023-11-24T06:25:49.505313+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:25:49.517915+0000","last_undegraded":"2023-11-24T06:25:49.517915+0000","last_fullsized":"2023-11-24T06:25:49.517915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T06:25:49.537908+0000","last_change":"2023-11-24T06:25:49.533010+0000","last_active":"2023-11-24T06:25:49.537908+0000","last_peered":"2023-11-24T06:25:49.537908+0000","last_clean":"2023-11-24T06:25:49.537908+0000","last_became_active":"2023-11-24T06:25:49.532766+0000","last_became_peered":"2023-11-24T06:25:49.532766+0000","last_unstale":"2023-11-24T06:25:49.537908+0000","last_undegraded":"2023-11-24T06:25:49.537908+0000","last_fullsized":"2023-11-24T06:25:49.537908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:13.907697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-24T06:25:48.509286+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:48.509286+0000","last_peered":"2023-11-24T06:25:48.509286+0000","last_clean":"2023-11-24T06:25:48.509286+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:48.509286+0000","last_undegraded":"2023-11-24T06:25:48.509286+0000","last_fullsized":"2023-11-24T06:25:48.509286+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.495}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.439},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.747}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.315}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.371}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.747}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.379}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.752}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.89400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.499},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.51100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.441},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.625},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515714+0000', 'last_change': '2023-11-24T06:25:49.515714+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515714+0000', 'last_undegraded': '2023-11-24T06:25:49.515714+0000', 'last_fullsized': '2023-11-24T06:25:49.515714+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.522707+0000', 'last_change': '2023-11-24T06:25:49.522707+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.522707+0000', 'last_undegraded': '2023-11-24T06:25:49.522707+0000', 'last_fullsized': '2023-11-24T06:25:49.522707+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513933+0000', 'last_change': '2023-11-24T06:25:49.513933+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513933+0000', 'last_undegraded': '2023-11-24T06:25:49.513933+0000', 'last_fullsized': '2023-11-24T06:25:49.513933+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-11-24T06:25:55.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.521392+0000', 'last_change': '2023-11-24T06:25:49.521392+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.521392+0000', 'last_undegraded': '2023-11-24T06:25:49.521392+0000', 'last_fullsized': '2023-11-24T06:25:49.521392+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.522718+0000', 'last_change': '2023-11-24T06:25:49.522718+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.522718+0000', 'last_undegraded': '2023-11-24T06:25:49.522718+0000', 'last_fullsized': '2023-11-24T06:25:49.522718+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.526883+0000', 'last_change': '2023-11-24T06:25:49.526883+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.526883+0000', 'last_undegraded': '2023-11-24T06:25:49.526883+0000', 'last_fullsized': '2023-11-24T06:25:49.526883+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516210+0000', 'last_change': '2023-11-24T06:25:49.516210+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516210+0000', 'last_undegraded': '2023-11-24T06:25:49.516210+0000', 'last_fullsized': '2023-11-24T06:25:49.516210+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.521856+0000', 'last_change': '2023-11-24T06:25:49.521856+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.521856+0000', 'last_undegraded': '2023-11-24T06:25:49.521856+0000', 'last_fullsized': '2023-11-24T06:25:49.521856+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-11-24T06:25:55.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.520072+0000', 'last_change': '2023-11-24T06:25:49.520072+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.520072+0000', 'last_undegraded': '2023-11-24T06:25:49.520072+0000', 'last_fullsized': '2023-11-24T06:25:49.520072+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.520473+0000', 'last_change': '2023-11-24T06:25:49.520473+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.520473+0000', 'last_undegraded': '2023-11-24T06:25:49.520473+0000', 'last_fullsized': '2023-11-24T06:25:49.520473+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.519995+0000', 'last_change': '2023-11-24T06:25:49.519995+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.519995+0000', 'last_undegraded': '2023-11-24T06:25:49.519995+0000', 'last_fullsized': '2023-11-24T06:25:49.519995+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.524278+0000', 'last_change': '2023-11-24T06:25:49.524278+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.524278+0000', 'last_undegraded': '2023-11-24T06:25:49.524278+0000', 'last_fullsized': '2023-11-24T06:25:49.524278+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517000+0000', 'last_change': '2023-11-24T06:25:49.517000+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517000+0000', 'last_undegraded': '2023-11-24T06:25:49.517000+0000', 'last_fullsized': '2023-11-24T06:25:49.517000+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-11-24T06:25:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.520005+0000', 'last_change': '2023-11-24T06:25:49.520005+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.520005+0000', 'last_undegraded': '2023-11-24T06:25:49.520005+0000', 'last_fullsized': '2023-11-24T06:25:49.520005+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.525682+0000', 'last_change': '2023-11-24T06:25:49.525682+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.525682+0000', 'last_undegraded': '2023-11-24T06:25:49.525682+0000', 'last_fullsized': '2023-11-24T06:25:49.525682+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517415+0000', 'last_change': '2023-11-24T06:25:49.517415+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517415+0000', 'last_undegraded': '2023-11-24T06:25:49.517415+0000', 'last_fullsized': '2023-11-24T06:25:49.517415+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513815+0000', 'last_change': '2023-11-24T06:25:49.513815+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513815+0000', 'last_undegraded': '2023-11-24T06:25:49.513815+0000', 'last_fullsized': '2023-11-24T06:25:49.513815+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515328+0000', 'last_change': '2023-11-24T06:25:49.515328+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515328+0000', 'last_undegraded': '2023-11-24T06:25:49.515328+0000', 'last_fullsized': '2023-11-24T06:25:49.515328+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-11-24T06:25:55.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518515+0000', 'last_change': '2023-11-24T06:25:49.518515+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518515+0000', 'last_undegraded': '2023-11-24T06:25:49.518515+0000', 'last_fullsized': '2023-11-24T06:25:49.518515+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516635+0000', 'last_change': '2023-11-24T06:25:49.516635+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516635+0000', 'last_undegraded': '2023-11-24T06:25:49.516635+0000', 'last_fullsized': '2023-11-24T06:25:49.516635+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.527437+0000', 'last_change': '2023-11-24T06:25:49.527437+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.527437+0000', 'last_undegraded': '2023-11-24T06:25:49.527437+0000', 'last_fullsized': '2023-11-24T06:25:49.527437+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.519517+0000', 'last_change': '2023-11-24T06:25:49.519517+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.519517+0000', 'last_undegraded': '2023-11-24T06:25:49.519517+0000', 'last_fullsized': '2023-11-24T06:25:49.519517+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518630+0000', 'last_change': '2023-11-24T06:25:49.518630+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518630+0000', 'last_undegraded': '2023-11-24T06:25:49.518630+0000', 'last_fullsized': '2023-11-24T06:25:49.518630+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-11-24T06:25:55.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516923+0000', 'last_change': '2023-11-24T06:25:49.516923+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516923+0000', 'last_undegraded': '2023-11-24T06:25:49.516923+0000', 'last_fullsized': '2023-11-24T06:25:49.516923+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517579+0000', 'last_change': '2023-11-24T06:25:49.517579+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517579+0000', 'last_undegraded': '2023-11-24T06:25:49.517579+0000', 'last_fullsized': '2023-11-24T06:25:49.517579+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516556+0000', 'last_change': '2023-11-24T06:25:49.516556+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516556+0000', 'last_undegraded': '2023-11-24T06:25:49.516556+0000', 'last_fullsized': '2023-11-24T06:25:49.516556+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513365+0000', 'last_change': '2023-11-24T06:25:49.513365+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513365+0000', 'last_undegraded': '2023-11-24T06:25:49.513365+0000', 'last_fullsized': '2023-11-24T06:25:49.513365+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.525130+0000', 'last_change': '2023-11-24T06:25:49.525130+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.525130+0000', 'last_undegraded': '2023-11-24T06:25:49.525130+0000', 'last_fullsized': '2023-11-24T06:25:49.525130+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-11-24T06:25:55.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516966+0000', 'last_change': '2023-11-24T06:25:49.516966+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516966+0000', 'last_undegraded': '2023-11-24T06:25:49.516966+0000', 'last_fullsized': '2023-11-24T06:25:49.516966+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.514385+0000', 'last_change': '2023-11-24T06:25:49.514385+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.514385+0000', 'last_undegraded': '2023-11-24T06:25:49.514385+0000', 'last_fullsized': '2023-11-24T06:25:49.514385+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.525215+0000', 'last_change': '2023-11-24T06:25:49.525215+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.525215+0000', 'last_undegraded': '2023-11-24T06:25:49.525215+0000', 'last_fullsized': '2023-11-24T06:25:49.525215+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.516434+0000', 'last_change': '2023-11-24T06:25:49.516434+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.516434+0000', 'last_undegraded': '2023-11-24T06:25:49.516434+0000', 'last_fullsized': '2023-11-24T06:25:49.516434+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.519016+0000', 'last_change': '2023-11-24T06:25:49.519016+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.519016+0000', 'last_undegraded': '2023-11-24T06:25:49.519016+0000', 'last_fullsized': '2023-11-24T06:25:49.519016+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-24T06:25:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.519849+0000', 'last_change': '2023-11-24T06:25:49.519849+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.519849+0000', 'last_undegraded': '2023-11-24T06:25:49.519849+0000', 'last_fullsized': '2023-11-24T06:25:49.519849+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.524684+0000', 'last_change': '2023-11-24T06:25:49.524684+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.524684+0000', 'last_undegraded': '2023-11-24T06:25:49.524684+0000', 'last_fullsized': '2023-11-24T06:25:49.524684+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518734+0000', 'last_change': '2023-11-24T06:25:49.518734+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518734+0000', 'last_undegraded': '2023-11-24T06:25:49.518734+0000', 'last_fullsized': '2023-11-24T06:25:49.518734+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513919+0000', 'last_change': '2023-11-24T06:25:49.513919+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513919+0000', 'last_undegraded': '2023-11-24T06:25:49.513919+0000', 'last_fullsized': '2023-11-24T06:25:49.513919+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.520725+0000', 'last_change': '2023-11-24T06:25:49.520725+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.520725+0000', 'last_undegraded': '2023-11-24T06:25:49.520725+0000', 'last_fullsized': '2023-11-24T06:25:49.520725+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-24T06:25:55.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.519532+0000', 'last_change': '2023-11-24T06:25:49.519532+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.519532+0000', 'last_undegraded': '2023-11-24T06:25:49.519532+0000', 'last_fullsized': '2023-11-24T06:25:49.519532+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517845+0000', 'last_change': '2023-11-24T06:25:49.517845+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517845+0000', 'last_undegraded': '2023-11-24T06:25:49.517845+0000', 'last_fullsized': '2023-11-24T06:25:49.517845+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513631+0000', 'last_change': '2023-11-24T06:25:49.513631+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513631+0000', 'last_undegraded': '2023-11-24T06:25:49.513631+0000', 'last_fullsized': '2023-11-24T06:25:49.513631+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515185+0000', 'last_change': '2023-11-24T06:25:49.515185+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515185+0000', 'last_undegraded': '2023-11-24T06:25:49.515185+0000', 'last_fullsized': '2023-11-24T06:25:49.515185+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.525547+0000', 'last_change': '2023-11-24T06:25:49.525547+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.525547+0000', 'last_undegraded': '2023-11-24T06:25:49.525547+0000', 'last_fullsized': '2023-11-24T06:25:49.525547+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-24T06:25:55.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515945+0000', 'last_change': '2023-11-24T06:25:49.515945+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515945+0000', 'last_undegraded': '2023-11-24T06:25:49.515945+0000', 'last_fullsized': '2023-11-24T06:25:49.515945+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517033+0000', 'last_change': '2023-11-24T06:25:49.517033+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517033+0000', 'last_undegraded': '2023-11-24T06:25:49.517033+0000', 'last_fullsized': '2023-11-24T06:25:49.517033+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.520241+0000', 'last_change': '2023-11-24T06:25:49.520241+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.520241+0000', 'last_undegraded': '2023-11-24T06:25:49.520241+0000', 'last_fullsized': '2023-11-24T06:25:49.520241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.524855+0000', 'last_change': '2023-11-24T06:25:49.524855+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.524855+0000', 'last_undegraded': '2023-11-24T06:25:49.524855+0000', 'last_fullsized': '2023-11-24T06:25:49.524855+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515749+0000', 'last_change': '2023-11-24T06:25:49.515749+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515749+0000', 'last_undegraded': '2023-11-24T06:25:49.515749+0000', 'last_fullsized': '2023-11-24T06:25:49.515749+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515868+0000', 'last_change': '2023-11-24T06:25:49.515868+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515868+0000', 'last_undegraded': '2023-11-24T06:25:49.515868+0000', 'last_fullsized': '2023-11-24T06:25:49.515868+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518386+0000', 'last_change': '2023-11-24T06:25:49.518386+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518386+0000', 'last_undegraded': '2023-11-24T06:25:49.518386+0000', 'last_fullsized': '2023-11-24T06:25:49.518386+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517595+0000', 'last_change': '2023-11-24T06:25:49.517595+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517595+0000', 'last_undegraded': '2023-11-24T06:25:49.517595+0000', 'last_fullsized': '2023-11-24T06:25:49.517595+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518084+0000', 'last_change': '2023-11-24T06:25:49.518084+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518084+0000', 'last_undegraded': '2023-11-24T06:25:49.518084+0000', 'last_fullsized': '2023-11-24T06:25:49.518084+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.525610+0000', 'last_change': '2023-11-24T06:25:49.525610+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.525610+0000', 'last_undegraded': '2023-11-24T06:25:49.525610+0000', 'last_fullsized': '2023-11-24T06:25:49.525610+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-24T06:25:55.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518503+0000', 'last_change': '2023-11-24T06:25:49.518503+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518503+0000', 'last_undegraded': '2023-11-24T06:25:49.518503+0000', 'last_fullsized': '2023-11-24T06:25:49.518503+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515808+0000', 'last_change': '2023-11-24T06:25:49.515808+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515808+0000', 'last_undegraded': '2023-11-24T06:25:49.515808+0000', 'last_fullsized': '2023-11-24T06:25:49.515808+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518778+0000', 'last_change': '2023-11-24T06:25:49.518778+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518778+0000', 'last_undegraded': '2023-11-24T06:25:49.518778+0000', 'last_fullsized': '2023-11-24T06:25:49.518778+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.518028+0000', 'last_change': '2023-11-24T06:25:49.518028+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.518028+0000', 'last_undegraded': '2023-11-24T06:25:49.518028+0000', 'last_fullsized': '2023-11-24T06:25:49.518028+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517885+0000', 'last_change': '2023-11-24T06:25:49.517885+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517885+0000', 'last_undegraded': '2023-11-24T06:25:49.517885+0000', 'last_fullsized': '2023-11-24T06:25:49.517885+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-24T06:25:55.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.515368+0000', 'last_change': '2023-11-24T06:25:49.515368+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.515368+0000', 'last_undegraded': '2023-11-24T06:25:49.515368+0000', 'last_fullsized': '2023-11-24T06:25:49.515368+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.524885+0000', 'last_change': '2023-11-24T06:25:49.524885+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.524885+0000', 'last_undegraded': '2023-11-24T06:25:49.524885+0000', 'last_fullsized': '2023-11-24T06:25:49.524885+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.526406+0000', 'last_change': '2023-11-24T06:25:49.526406+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.526406+0000', 'last_undegraded': '2023-11-24T06:25:49.526406+0000', 'last_fullsized': '2023-11-24T06:25:49.526406+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.523607+0000', 'last_change': '2023-11-24T06:25:49.523607+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.523607+0000', 'last_undegraded': '2023-11-24T06:25:49.523607+0000', 'last_fullsized': '2023-11-24T06:25:49.523607+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.513176+0000', 'last_change': '2023-11-24T06:25:49.513176+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.513176+0000', 'last_undegraded': '2023-11-24T06:25:49.513176+0000', 'last_fullsized': '2023-11-24T06:25:49.513176+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-24T06:25:55.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:25:49.517915+0000', 'last_change': '2023-11-24T06:25:49.517915+0000', 'last_active': '2023-11-24T06:25:49.505313+0000', 'last_peered': '2023-11-24T06:25:49.505313+0000', 'last_clean': '2023-11-24T06:25:49.505313+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:25:49.517915+0000', 'last_undegraded': '2023-11-24T06:25:49.517915+0000', 'last_fullsized': '2023-11-24T06:25:49.517915+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'last_clean_scrub_stamp': '2023-11-24T06:25:49.505313+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:25:55.418 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T06:25:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.556+0000 7f78559fd700 1 Processor -- start 2023-11-24T06:25:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.556+0000 7f78559fd700 1 -- start start 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f78559fd700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78501446a0 0x7f78500b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f78559fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f785006c710 0x7f78500b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f78559fd700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78500af950 con 0x7f78500adf00 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f78559fd700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7850144a80 con 0x7f78501446a0 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f78559fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78500afc30 con 0x7f785006c710 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784e7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f78500adf00 0x7f785006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56624/0 (socket says 172.21.15.114:56624) 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784e7fc700 1 -- 172.21.15.114:0/176714064 learned_addr learned my addr 172.21.15.114:0/176714064 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784effd700 1 --2- 172.21.15.114:0/176714064 >> v2:172.21.15.172:3300/0 conn(0x7f78501446a0 0x7f78500b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784f7fe700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f785006c710 0x7f78500b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784dffb700 1 -- 172.21.15.114:0/176714064 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348227291 0 0) 0x7f78500af950 con 0x7f78500adf00 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784dffb700 1 -- 172.21.15.114:0/176714064 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78380035b0 con 0x7f78500adf00 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784effd700 1 -- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f785006c710 msgr2=0x7f78500b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784effd700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f785006c710 0x7f78500b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784effd700 1 -- 172.21.15.114:0/176714064 >> v1:172.21.15.114:6789/0 conn(0x7f78500adf00 legacy=0x7f785006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.557+0000 7f784effd700 1 -- 172.21.15.114:0/176714064 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78500aff10 con 0x7f78501446a0 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f784effd700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f78500b5ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7840009840 tx=0x7f7840009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d2d0f8dbe4229 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f784dffb700 1 -- 172.21.15.114:0/176714064 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840011830 con 0x7f78501446a0 2023-11-24T06:25:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f784dffb700 1 -- 172.21.15.114:0/176714064 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7840011990 con 0x7f78501446a0 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f784dffb700 1 -- 172.21.15.114:0/176714064 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840011d20 con 0x7f78501446a0 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 -- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 msgr2=0x7f78500b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f78500b5ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7840009840 tx=0x7f7840009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 -- 172.21.15.114:0/176714064 shutdown_connections 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f785006c710 0x7f78500b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 --2- 172.21.15.114:0/176714064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f78500b5ae0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 -- 172.21.15.114:0/176714064 >> 172.21.15.114:0/176714064 conn(0x7f7850071b90 msgr2=0x7f7850071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 -- 172.21.15.114:0/176714064 shutdown_connections 2023-11-24T06:25:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.558+0000 7f78559fd700 1 -- 172.21.15.114:0/176714064 wait complete. 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.560+0000 7f78559fd700 1 Processor -- start 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.560+0000 7f78559fd700 1 -- start start 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500adf00 0x7f785006df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f785006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785009bbd0 con 0x7f785006c710 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785013ab80 con 0x7f78501446a0 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785009b000 con 0x7f78500adf00 2023-11-24T06:25:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40920/0 (socket says 172.21.15.114:40920) 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 -- 172.21.15.114:0/335845031 learned_addr learned my addr 172.21.15.114:0/335845031 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784f7fe700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f785006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784e7fc700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500adf00 0x7f785006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 -- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500adf00 msgr2=0x7f785006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500adf00 0x7f785006df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 -- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 msgr2=0x7f785006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f785006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 -- 172.21.15.114:0/335845031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400094a0 con 0x7f785006c710 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f784effd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78400090b0 tx=0x7f7840004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840004590 con 0x7f785006c710 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784001e070 con 0x7f785006c710 2023-11-24T06:25:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785009b2a0 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400038e0 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.561+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785009b860 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.562+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78500753f0 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.563+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f7840003a40 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.563+0000 7f7837fff700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 0x7f783c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.563+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f784008a6a0 con 0x7f785006c710 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.563+0000 7f784e7fc700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 0x7f783c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.563+0000 7f784e7fc700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 0x7f783c06d540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7850074b80 tx=0x7f7844007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.569 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.567+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7840055560 con 0x7f785006c710 2023-11-24T06:25:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.837+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7850067f60 con 0x7f785006c710 2023-11-24T06:25:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.837+0000 7f7837fff700 1 -- 172.21.15.114:0/335845031 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1323 (secure 0 0 0) 0x7f7840015540 con 0x7f785006c710 2023-11-24T06:25:55.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:55.840 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":123,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1700807043,"num_in_osds":8,"osd_in_since":1700807034,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":226496512,"bytes_avail":772867616768,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5646}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-24T06:25:11.718249+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 msgr2=0x7f783c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 0x7f783c06d540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7850074b80 tx=0x7f7844007dc0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 msgr2=0x7f785013a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78400090b0 tx=0x7f7840004750 comp rx=0 tx=0).stop 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 shutdown_connections 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f783c06b080 0x7f783c06d540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78501446a0 0x7f785006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f785006c710 0x7f785013a640 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 --2- 172.21.15.114:0/335845031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78500adf00 0x7f785006df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 >> 172.21.15.114:0/335845031 conn(0x7f7850071b90 msgr2=0x7f78500b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 shutdown_connections 2023-11-24T06:25:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.839+0000 7f78559fd700 1 -- 172.21.15.114:0/335845031 wait complete. 2023-11-24T06:25:55.854 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T06:25:55.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:55.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.972+0000 7fba323b0700 1 Processor -- start 2023-11-24T06:25:55.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.972+0000 7fba323b0700 1 -- start start 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba323b0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fba2c0b0bb0 0x7fba2c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba323b0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0af8c0 con 0x7fba2c0b1940 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba323b0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0b1030 con 0x7fba2c15afc0 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba323b0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0b0790 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fba2c0b0bb0 0x7fba2c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba3094d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fba2c0b1940 0x7fba2c0b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56640/0 (socket says 172.21.15.114:56640) 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba3094d700 1 -- 172.21.15.114:0/1659859125 learned_addr learned my addr 172.21.15.114:0/1659859125 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2b7fe700 1 -- 172.21.15.114:0/1659859125 >> v1:172.21.15.172:6789/0 conn(0x7fba2c15afc0 legacy=0x7fba2c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2b7fe700 1 -- 172.21.15.114:0/1659859125 >> v1:172.21.15.114:6789/0 conn(0x7fba2c0b1940 legacy=0x7fba2c0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2affd700 1 -- 172.21.15.114:0/1659859125 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752439483 0 0) 0x7fba2c0af8c0 con 0x7fba2c0b1940 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2b7fe700 1 -- 172.21.15.114:0/1659859125 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0afb10 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2b7fe700 1 --2- 172.21.15.114:0/1659859125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b0bb0 0x7fba2c0b1400 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009840 tx=0x7fba1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bef8f7a49bf984 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2affd700 1 -- 172.21.15.114:0/1659859125 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c011830 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2affd700 1 -- 172.21.15.114:0/1659859125 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c011990 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.973+0000 7fba2affd700 1 -- 172.21.15.114:0/1659859125 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c011d20 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 -- 172.21.15.114:0/1659859125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b0bb0 msgr2=0x7fba2c0b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 --2- 172.21.15.114:0/1659859125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b0bb0 0x7fba2c0b1400 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009840 tx=0x7fba1c009b50 comp rx=0 tx=0).stop 2023-11-24T06:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 -- 172.21.15.114:0/1659859125 shutdown_connections 2023-11-24T06:25:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 --2- 172.21.15.114:0/1659859125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b0bb0 0x7fba2c0b1400 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 -- 172.21.15.114:0/1659859125 >> 172.21.15.114:0/1659859125 conn(0x7fba2c0b57b0 msgr2=0x7fba2c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 -- 172.21.15.114:0/1659859125 shutdown_connections 2023-11-24T06:25:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.974+0000 7fba323b0700 1 -- 172.21.15.114:0/1659859125 wait complete. 2023-11-24T06:25:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.975+0000 7fba323b0700 1 Processor -- start 2023-11-24T06:25:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.975+0000 7fba323b0700 1 -- start start 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba2c0b0bb0 0x7fba2c0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba2c15afc0 0x7fba2c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0cb040 con 0x7fba2c0b1940 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1f7590 con 0x7fba2c15afc0 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba323b0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1f7870 con 0x7fba2c0b0bb0 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba2b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40936/0 (socket says 172.21.15.114:40936) 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba2b7fe700 1 -- 172.21.15.114:0/2980573551 learned_addr learned my addr 172.21.15.114:0/2980573551 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba3094d700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba2c15afc0 0x7fba2c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.976+0000 7fba2bfff700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba2c0b0bb0 0x7fba2c0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 -- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba2c0b0bb0 msgr2=0x7fba2c0ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba2c0b0bb0 0x7fba2c0ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 -- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba2c15afc0 msgr2=0x7fba2c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba2c15afc0 0x7fba2c0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 -- 172.21.15.114:0/2980573551 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c0094a0 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba2b7fe700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009b90 tx=0x7fba1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0036a0 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c01e070 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c1f7b50 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c003a00 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.977+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c1f8110 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.978+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c067f60 con 0x7fba2c0b1940 2023-11-24T06:25:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.978+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fba1c003b60 con 0x7fba2c0b1940 2023-11-24T06:25:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.979+0000 7fba28ff9700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 0x7fba1806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.979+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba1c08a9e0 con 0x7fba2c0b1940 2023-11-24T06:25:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.979+0000 7fba2bfff700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 0x7fba1806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.979+0000 7fba2bfff700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 0x7fba1806d540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba20000f60 tx=0x7fba20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:55.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:55.983+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba1c0558a0 con 0x7fba2c0b1940 2023-11-24T06:25:56.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.169+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba2c0b00d0 con 0x7fba1806b080 2023-11-24T06:25:56.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.177+0000 7fba28ff9700 1 -- 172.21.15.114:0/2980573551 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288760 (secure 0 0 0) 0x7fba20001490 con 0x7fba1806b080 2023-11-24T06:25:56.184 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 msgr2=0x7fba1806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 0x7fba1806d540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba20000f60 tx=0x7fba20009040 comp rx=0 tx=0).stop 2023-11-24T06:25:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 msgr2=0x7fba2c0aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009b90 tx=0x7fba1c004750 comp rx=0 tx=0).stop 2023-11-24T06:25:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 shutdown_connections 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fba1806b080 0x7fba1806d540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba2c15afc0 0x7fba2c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba2c0b0bb0 0x7fba2c0ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 --2- 172.21.15.114:0/2980573551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba2c0b1940 0x7fba2c0aed40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.180+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 >> 172.21.15.114:0/2980573551 conn(0x7fba2c0b57b0 msgr2=0x7fba2c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.181+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 shutdown_connections 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:56.181+0000 7fba323b0700 1 -- 172.21.15.114:0/2980573551 wait complete. 2023-11-24T06:25:56.187 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:56.196 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-24T06:25:55.726267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221540,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753180,"statfs":{"total":773094113280,"available":772867256320,"internally_reserved":0,"allocated":5480448,"data_stored":2901132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.215763"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529438+0000","last_change":"2023-11-24T06:25:50.531873+0000","last_active":"2023-11-24T06:25:51.529438+0000","last_peered":"2023-11-24T06:25:51.529438+0000","last_clean":"2023-11-24T06:25:51.529438+0000","last_became_active":"2023-11-24T06:25:50.531378+0000","last_became_peered":"2023-11-24T06:25:50.531378+0000","last_unstale":"2023-11-24T06:25:51.529438+0000","last_undegraded":"2023-11-24T06:25:51.529438+0000","last_fullsized":"2023-11-24T06:25:51.529438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:27.438992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770954+0000","last_change":"2023-11-24T06:25:50.546143+0000","last_active":"2023-11-24T06:25:51.770954+0000","last_peered":"2023-11-24T06:25:51.770954+0000","last_clean":"2023-11-24T06:25:51.770954+0000","last_became_active":"2023-11-24T06:25:50.545827+0000","last_became_peered":"2023-11-24T06:25:50.545827+0000","last_unstale":"2023-11-24T06:25:51.770954+0000","last_undegraded":"2023-11-24T06:25:51.770954+0000","last_fullsized":"2023-11-24T06:25:51.770954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:39.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385041+0000","last_change":"2023-11-24T06:25:50.546400+0000","last_active":"2023-11-24T06:25:52.385041+0000","last_peered":"2023-11-24T06:25:52.385041+0000","last_clean":"2023-11-24T06:25:52.385041+0000","last_became_active":"2023-11-24T06:25:50.546094+0000","last_became_peered":"2023-11-24T06:25:50.546094+0000","last_unstale":"2023-11-24T06:25:52.385041+0000","last_undegraded":"2023-11-24T06:25:52.385041+0000","last_fullsized":"2023-11-24T06:25:52.385041+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:14.915316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528123+0000","last_change":"2023-11-24T06:25:50.543369+0000","last_active":"2023-11-24T06:25:51.528123+0000","last_peered":"2023-11-24T06:25:51.528123+0000","last_clean":"2023-11-24T06:25:51.528123+0000","last_became_active":"2023-11-24T06:25:50.543047+0000","last_became_peered":"2023-11-24T06:25:50.543047+0000","last_unstale":"2023-11-24T06:25:51.528123+0000","last_undegraded":"2023-11-24T06:25:51.528123+0000","last_fullsized":"2023-11-24T06:25:51.528123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:59.859400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770776+0000","last_change":"2023-11-24T06:25:50.545946+0000","last_active":"2023-11-24T06:25:51.770776+0000","last_peered":"2023-11-24T06:25:51.770776+0000","last_clean":"2023-11-24T06:25:51.770776+0000","last_became_active":"2023-11-24T06:25:50.545667+0000","last_became_peered":"2023-11-24T06:25:50.545667+0000","last_unstale":"2023-11-24T06:25:51.770776+0000","last_undegraded":"2023-11-24T06:25:51.770776+0000","last_fullsized":"2023-11-24T06:25:51.770776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:44.770366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385273+0000","last_change":"2023-11-24T06:25:50.546095+0000","last_active":"2023-11-24T06:25:52.385273+0000","last_peered":"2023-11-24T06:25:52.385273+0000","last_clean":"2023-11-24T06:25:52.385273+0000","last_became_active":"2023-11-24T06:25:50.545799+0000","last_became_peered":"2023-11-24T06:25:50.545799+0000","last_unstale":"2023-11-24T06:25:52.385273+0000","last_undegraded":"2023-11-24T06:25:52.385273+0000","last_fullsized":"2023-11-24T06:25:52.385273+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:00.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528438+0000","last_change":"2023-11-24T06:25:50.548753+0000","last_active":"2023-11-24T06:25:51.528438+0000","last_peered":"2023-11-24T06:25:51.528438+0000","last_clean":"2023-11-24T06:25:51.528438+0000","last_became_active":"2023-11-24T06:25:50.548478+0000","last_became_peered":"2023-11-24T06:25:50.548478+0000","last_unstale":"2023-11-24T06:25:51.528438+0000","last_undegraded":"2023-11-24T06:25:51.528438+0000","last_fullsized":"2023-11-24T06:25:51.528438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:41.853651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528221+0000","last_change":"2023-11-24T06:25:50.544689+0000","last_active":"2023-11-24T06:25:51.528221+0000","last_peered":"2023-11-24T06:25:51.528221+0000","last_clean":"2023-11-24T06:25:51.528221+0000","last_became_active":"2023-11-24T06:25:50.544431+0000","last_became_peered":"2023-11-24T06:25:50.544431+0000","last_unstale":"2023-11-24T06:25:51.528221+0000","last_undegraded":"2023-11-24T06:25:51.528221+0000","last_fullsized":"2023-11-24T06:25:51.528221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:43.446376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528194+0000","last_change":"2023-11-24T06:25:50.550332+0000","last_active":"2023-11-24T06:25:51.528194+0000","last_peered":"2023-11-24T06:25:51.528194+0000","last_clean":"2023-11-24T06:25:51.528194+0000","last_became_active":"2023-11-24T06:25:50.548627+0000","last_became_peered":"2023-11-24T06:25:50.548627+0000","last_unstale":"2023-11-24T06:25:51.528194+0000","last_undegraded":"2023-11-24T06:25:51.528194+0000","last_fullsized":"2023-11-24T06:25:51.528194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:35.110793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528193+0000","last_change":"2023-11-24T06:25:50.543547+0000","last_active":"2023-11-24T06:25:51.528193+0000","last_peered":"2023-11-24T06:25:51.528193+0000","last_clean":"2023-11-24T06:25:51.528193+0000","last_became_active":"2023-11-24T06:25:50.543257+0000","last_became_peered":"2023-11-24T06:25:50.543257+0000","last_unstale":"2023-11-24T06:25:51.528193+0000","last_undegraded":"2023-11-24T06:25:51.528193+0000","last_fullsized":"2023-11-24T06:25:51.528193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:51.509785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528303+0000","last_change":"2023-11-24T06:25:50.554529+0000","last_active":"2023-11-24T06:25:51.528303+0000","last_peered":"2023-11-24T06:25:51.528303+0000","last_clean":"2023-11-24T06:25:51.528303+0000","last_became_active":"2023-11-24T06:25:50.554245+0000","last_became_peered":"2023-11-24T06:25:50.554245+0000","last_unstale":"2023-11-24T06:25:51.528303+0000","last_undegraded":"2023-11-24T06:25:51.528303+0000","last_fullsized":"2023-11-24T06:25:51.528303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:44.770366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770851+0000","last_change":"2023-11-24T06:25:50.545580+0000","last_active":"2023-11-24T06:25:51.770851+0000","last_peered":"2023-11-24T06:25:51.770851+0000","last_clean":"2023-11-24T06:25:51.770851+0000","last_became_active":"2023-11-24T06:25:50.545280+0000","last_became_peered":"2023-11-24T06:25:50.545280+0000","last_unstale":"2023-11-24T06:25:51.770851+0000","last_undegraded":"2023-11-24T06:25:51.770851+0000","last_fullsized":"2023-11-24T06:25:51.770851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:41.853651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903795+0000","last_change":"2023-11-24T06:25:50.558908+0000","last_active":"2023-11-24T06:25:51.903795+0000","last_peered":"2023-11-24T06:25:51.903795+0000","last_clean":"2023-11-24T06:25:51.903795+0000","last_became_active":"2023-11-24T06:25:50.558628+0000","last_became_peered":"2023-11-24T06:25:50.558628+0000","last_unstale":"2023-11-24T06:25:51.903795+0000","last_undegraded":"2023-11-24T06:25:51.903795+0000","last_fullsized":"2023-11-24T06:25:51.903795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:19.029120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527910+0000","last_change":"2023-11-24T06:25:50.554035+0000","last_active":"2023-11-24T06:25:51.527910+0000","last_peered":"2023-11-24T06:25:51.527910+0000","last_clean":"2023-11-24T06:25:51.527910+0000","last_became_active":"2023-11-24T06:25:50.553744+0000","last_became_peered":"2023-11-24T06:25:50.553744+0000","last_unstale":"2023-11-24T06:25:51.527910+0000","last_undegraded":"2023-11-24T06:25:51.527910+0000","last_fullsized":"2023-11-24T06:25:51.527910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:44.191847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385128+0000","last_change":"2023-11-24T06:25:50.547859+0000","last_active":"2023-11-24T06:25:52.385128+0000","last_peered":"2023-11-24T06:25:52.385128+0000","last_clean":"2023-11-24T06:25:52.385128+0000","last_became_active":"2023-11-24T06:25:50.546438+0000","last_became_peered":"2023-11-24T06:25:50.546438+0000","last_unstale":"2023-11-24T06:25:52.385128+0000","last_undegraded":"2023-11-24T06:25:52.385128+0000","last_fullsized":"2023-11-24T06:25:52.385128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:51.032214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528282+0000","last_change":"2023-11-24T06:25:50.543140+0000","last_active":"2023-11-24T06:25:51.528282+0000","last_peered":"2023-11-24T06:25:51.528282+0000","last_clean":"2023-11-24T06:25:51.528282+0000","last_became_active":"2023-11-24T06:25:50.542818+0000","last_became_peered":"2023-11-24T06:25:50.542818+0000","last_unstale":"2023-11-24T06:25:51.528282+0000","last_undegraded":"2023-11-24T06:25:51.528282+0000","last_fullsized":"2023-11-24T06:25:51.528282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.373959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385134+0000","last_change":"2023-11-24T06:25:50.539395+0000","last_active":"2023-11-24T06:25:52.385134+0000","last_peered":"2023-11-24T06:25:52.385134+0000","last_clean":"2023-11-24T06:25:52.385134+0000","last_became_active":"2023-11-24T06:25:50.539030+0000","last_became_peered":"2023-11-24T06:25:50.539030+0000","last_unstale":"2023-11-24T06:25:52.385134+0000","last_undegraded":"2023-11-24T06:25:52.385134+0000","last_fullsized":"2023-11-24T06:25:52.385134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:08.442324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385190+0000","last_change":"2023-11-24T06:25:50.534513+0000","last_active":"2023-11-24T06:25:52.385190+0000","last_peered":"2023-11-24T06:25:52.385190+0000","last_clean":"2023-11-24T06:25:52.385190+0000","last_became_active":"2023-11-24T06:25:50.534187+0000","last_became_peered":"2023-11-24T06:25:50.534187+0000","last_unstale":"2023-11-24T06:25:52.385190+0000","last_undegraded":"2023-11-24T06:25:52.385190+0000","last_fullsized":"2023-11-24T06:25:52.385190+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:06.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529501+0000","last_change":"2023-11-24T06:25:50.543322+0000","last_active":"2023-11-24T06:25:51.529501+0000","last_peered":"2023-11-24T06:25:51.529501+0000","last_clean":"2023-11-24T06:25:51.529501+0000","last_became_active":"2023-11-24T06:25:50.543067+0000","last_became_peered":"2023-11-24T06:25:50.543067+0000","last_unstale":"2023-11-24T06:25:51.529501+0000","last_undegraded":"2023-11-24T06:25:51.529501+0000","last_fullsized":"2023-11-24T06:25:51.529501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:29.296816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903990+0000","last_change":"2023-11-24T06:25:50.548979+0000","last_active":"2023-11-24T06:25:51.903990+0000","last_peered":"2023-11-24T06:25:51.903990+0000","last_clean":"2023-11-24T06:25:51.903990+0000","last_became_active":"2023-11-24T06:25:50.548725+0000","last_became_peered":"2023-11-24T06:25:50.548725+0000","last_unstale":"2023-11-24T06:25:51.903990+0000","last_undegraded":"2023-11-24T06:25:51.903990+0000","last_fullsized":"2023-11-24T06:25:51.903990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.379116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385507+0000","last_change":"2023-11-24T06:25:50.545765+0000","last_active":"2023-11-24T06:25:52.385507+0000","last_peered":"2023-11-24T06:25:52.385507+0000","last_clean":"2023-11-24T06:25:52.385507+0000","last_became_active":"2023-11-24T06:25:50.545446+0000","last_became_peered":"2023-11-24T06:25:50.545446+0000","last_unstale":"2023-11-24T06:25:52.385507+0000","last_undegraded":"2023-11-24T06:25:52.385507+0000","last_fullsized":"2023-11-24T06:25:52.385507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:56.447309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528065+0000","last_change":"2023-11-24T06:25:50.545609+0000","last_active":"2023-11-24T06:25:51.528065+0000","last_peered":"2023-11-24T06:25:51.528065+0000","last_clean":"2023-11-24T06:25:51.528065+0000","last_became_active":"2023-11-24T06:25:50.544865+0000","last_became_peered":"2023-11-24T06:25:50.544865+0000","last_unstale":"2023-11-24T06:25:51.528065+0000","last_undegraded":"2023-11-24T06:25:51.528065+0000","last_fullsized":"2023-11-24T06:25:51.528065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:42.713204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529210+0000","last_change":"2023-11-24T06:25:50.544181+0000","last_active":"2023-11-24T06:25:51.529210+0000","last_peered":"2023-11-24T06:25:51.529210+0000","last_clean":"2023-11-24T06:25:51.529210+0000","last_became_active":"2023-11-24T06:25:50.543881+0000","last_became_peered":"2023-11-24T06:25:50.543881+0000","last_unstale":"2023-11-24T06:25:51.529210+0000","last_undegraded":"2023-11-24T06:25:51.529210+0000","last_fullsized":"2023-11-24T06:25:51.529210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:13.713094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528681+0000","last_change":"2023-11-24T06:25:50.540318+0000","last_active":"2023-11-24T06:25:51.528681+0000","last_peered":"2023-11-24T06:25:51.528681+0000","last_clean":"2023-11-24T06:25:51.528681+0000","last_became_active":"2023-11-24T06:25:50.540040+0000","last_became_peered":"2023-11-24T06:25:50.540040+0000","last_unstale":"2023-11-24T06:25:51.528681+0000","last_undegraded":"2023-11-24T06:25:51.528681+0000","last_fullsized":"2023-11-24T06:25:51.528681+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:58.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529012+0000","last_change":"2023-11-24T06:25:50.558884+0000","last_active":"2023-11-24T06:25:51.529012+0000","last_peered":"2023-11-24T06:25:51.529012+0000","last_clean":"2023-11-24T06:25:51.529012+0000","last_became_active":"2023-11-24T06:25:50.558620+0000","last_became_peered":"2023-11-24T06:25:50.558620+0000","last_unstale":"2023-11-24T06:25:51.529012+0000","last_undegraded":"2023-11-24T06:25:51.529012+0000","last_fullsized":"2023-11-24T06:25:51.529012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:16.475409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903866+0000","last_change":"2023-11-24T06:25:50.555376+0000","last_active":"2023-11-24T06:25:51.903866+0000","last_peered":"2023-11-24T06:25:51.903866+0000","last_clean":"2023-11-24T06:25:51.903866+0000","last_became_active":"2023-11-24T06:25:50.555137+0000","last_became_peered":"2023-11-24T06:25:50.555137+0000","last_unstale":"2023-11-24T06:25:51.903866+0000","last_undegraded":"2023-11-24T06:25:51.903866+0000","last_fullsized":"2023-11-24T06:25:51.903866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:33.092309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385044+0000","last_change":"2023-11-24T06:25:50.540012+0000","last_active":"2023-11-24T06:25:52.385044+0000","last_peered":"2023-11-24T06:25:52.385044+0000","last_clean":"2023-11-24T06:25:52.385044+0000","last_became_active":"2023-11-24T06:25:50.539757+0000","last_became_peered":"2023-11-24T06:25:50.539757+0000","last_unstale":"2023-11-24T06:25:52.385044+0000","last_undegraded":"2023-11-24T06:25:52.385044+0000","last_fullsized":"2023-11-24T06:25:52.385044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:08.477759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.771040+0000","last_change":"2023-11-24T06:25:50.545665+0000","last_active":"2023-11-24T06:25:51.771040+0000","last_peered":"2023-11-24T06:25:51.771040+0000","last_clean":"2023-11-24T06:25:51.771040+0000","last_became_active":"2023-11-24T06:25:50.545297+0000","last_became_peered":"2023-11-24T06:25:50.545297+0000","last_unstale":"2023-11-24T06:25:51.771040+0000","last_undegraded":"2023-11-24T06:25:51.771040+0000","last_fullsized":"2023-11-24T06:25:51.771040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:43.722297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528256+0000","last_change":"2023-11-24T06:25:50.553740+0000","last_active":"2023-11-24T06:25:51.528256+0000","last_peered":"2023-11-24T06:25:51.528256+0000","last_clean":"2023-11-24T06:25:51.528256+0000","last_became_active":"2023-11-24T06:25:50.553475+0000","last_became_peered":"2023-11-24T06:25:50.553475+0000","last_unstale":"2023-11-24T06:25:51.528256+0000","last_undegraded":"2023-11-24T06:25:51.528256+0000","last_fullsized":"2023-11-24T06:25:51.528256+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:44.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385111+0000","last_change":"2023-11-24T06:25:50.531496+0000","last_active":"2023-11-24T06:25:52.385111+0000","last_peered":"2023-11-24T06:25:52.385111+0000","last_clean":"2023-11-24T06:25:52.385111+0000","last_became_active":"2023-11-24T06:25:50.531201+0000","last_became_peered":"2023-11-24T06:25:50.531201+0000","last_unstale":"2023-11-24T06:25:52.385111+0000","last_undegraded":"2023-11-24T06:25:52.385111+0000","last_fullsized":"2023-11-24T06:25:52.385111+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:44.191847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385322+0000","last_change":"2023-11-24T06:25:50.548666+0000","last_active":"2023-11-24T06:25:52.385322+0000","last_peered":"2023-11-24T06:25:52.385322+0000","last_clean":"2023-11-24T06:25:52.385322+0000","last_became_active":"2023-11-24T06:25:50.548138+0000","last_became_peered":"2023-11-24T06:25:50.548138+0000","last_unstale":"2023-11-24T06:25:52.385322+0000","last_undegraded":"2023-11-24T06:25:52.385322+0000","last_fullsized":"2023-11-24T06:25:52.385322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:45.473450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903738+0000","last_change":"2023-11-24T06:25:50.556726+0000","last_active":"2023-11-24T06:25:51.903738+0000","last_peered":"2023-11-24T06:25:51.903738+0000","last_clean":"2023-11-24T06:25:51.903738+0000","last_became_active":"2023-11-24T06:25:50.556409+0000","last_became_peered":"2023-11-24T06:25:50.556409+0000","last_unstale":"2023-11-24T06:25:51.903738+0000","last_undegraded":"2023-11-24T06:25:51.903738+0000","last_fullsized":"2023-11-24T06:25:51.903738+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:55.903587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385988+0000","last_change":"2023-11-24T06:25:50.552279+0000","last_active":"2023-11-24T06:25:52.385988+0000","last_peered":"2023-11-24T06:25:52.385988+0000","last_clean":"2023-11-24T06:25:52.385988+0000","last_became_active":"2023-11-24T06:25:50.552073+0000","last_became_peered":"2023-11-24T06:25:50.552073+0000","last_unstale":"2023-11-24T06:25:52.385988+0000","last_undegraded":"2023-11-24T06:25:52.385988+0000","last_fullsized":"2023-11-24T06:25:52.385988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:51.885765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563196+0000","last_change":"2023-11-24T06:25:49.531680+0000","last_active":"2023-11-24T06:25:50.563196+0000","last_peered":"2023-11-24T06:25:50.563196+0000","last_clean":"2023-11-24T06:25:50.563196+0000","last_became_active":"2023-11-24T06:25:49.531361+0000","last_became_peered":"2023-11-24T06:25:49.531361+0000","last_unstale":"2023-11-24T06:25:50.563196+0000","last_undegraded":"2023-11-24T06:25:50.563196+0000","last_fullsized":"2023-11-24T06:25:50.563196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:42.714678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528598+0000","last_change":"2023-11-24T06:25:50.542384+0000","last_active":"2023-11-24T06:25:51.528598+0000","last_peered":"2023-11-24T06:25:51.528598+0000","last_clean":"2023-11-24T06:25:51.528598+0000","last_became_active":"2023-11-24T06:25:50.542079+0000","last_became_peered":"2023-11-24T06:25:50.542079+0000","last_unstale":"2023-11-24T06:25:51.528598+0000","last_undegraded":"2023-11-24T06:25:51.528598+0000","last_fullsized":"2023-11-24T06:25:51.528598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:53.015676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.550312+0000","last_change":"2023-11-24T06:25:49.525389+0000","last_active":"2023-11-24T06:25:50.550312+0000","last_peered":"2023-11-24T06:25:50.550312+0000","last_clean":"2023-11-24T06:25:50.550312+0000","last_became_active":"2023-11-24T06:25:49.524856+0000","last_became_peered":"2023-11-24T06:25:49.524856+0000","last_unstale":"2023-11-24T06:25:50.550312+0000","last_undegraded":"2023-11-24T06:25:50.550312+0000","last_fullsized":"2023-11-24T06:25:50.550312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770935+0000","last_change":"2023-11-24T06:25:50.554666+0000","last_active":"2023-11-24T06:25:51.770935+0000","last_peered":"2023-11-24T06:25:51.770935+0000","last_clean":"2023-11-24T06:25:51.770935+0000","last_became_active":"2023-11-24T06:25:50.554392+0000","last_became_peered":"2023-11-24T06:25:50.554392+0000","last_unstale":"2023-11-24T06:25:51.770935+0000","last_undegraded":"2023-11-24T06:25:51.770935+0000","last_fullsized":"2023-11-24T06:25:51.770935+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:53:54.755693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.560598+0000","last_change":"2023-11-24T06:25:49.530053+0000","last_active":"2023-11-24T06:25:50.560598+0000","last_peered":"2023-11-24T06:25:50.560598+0000","last_clean":"2023-11-24T06:25:50.560598+0000","last_became_active":"2023-11-24T06:25:49.529544+0000","last_became_peered":"2023-11-24T06:25:49.529544+0000","last_unstale":"2023-11-24T06:25:50.560598+0000","last_undegraded":"2023-11-24T06:25:50.560598+0000","last_fullsized":"2023-11-24T06:25:50.560598+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:06.466292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385980+0000","last_change":"2023-11-24T06:25:50.552742+0000","last_active":"2023-11-24T06:25:52.385980+0000","last_peered":"2023-11-24T06:25:52.385980+0000","last_clean":"2023-11-24T06:25:52.385980+0000","last_became_active":"2023-11-24T06:25:50.549874+0000","last_became_peered":"2023-11-24T06:25:50.549874+0000","last_unstale":"2023-11-24T06:25:52.385980+0000","last_undegraded":"2023-11-24T06:25:52.385980+0000","last_fullsized":"2023-11-24T06:25:52.385980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:58.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.553637+0000","last_change":"2023-11-24T06:25:49.529413+0000","last_active":"2023-11-24T06:25:50.553637+0000","last_peered":"2023-11-24T06:25:50.553637+0000","last_clean":"2023-11-24T06:25:50.553637+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:50.553637+0000","last_undegraded":"2023-11-24T06:25:50.553637+0000","last_fullsized":"2023-11-24T06:25:50.553637+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.384929+0000","last_change":"2023-11-24T06:25:50.544430+0000","last_active":"2023-11-24T06:25:52.384929+0000","last_peered":"2023-11-24T06:25:52.384929+0000","last_clean":"2023-11-24T06:25:52.384929+0000","last_became_active":"2023-11-24T06:25:50.544143+0000","last_became_peered":"2023-11-24T06:25:50.544143+0000","last_unstale":"2023-11-24T06:25:52.384929+0000","last_undegraded":"2023-11-24T06:25:52.384929+0000","last_fullsized":"2023-11-24T06:25:52.384929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:58.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563127+0000","last_change":"2023-11-24T06:25:49.532801+0000","last_active":"2023-11-24T06:25:50.563127+0000","last_peered":"2023-11-24T06:25:50.563127+0000","last_clean":"2023-11-24T06:25:50.563127+0000","last_became_active":"2023-11-24T06:25:49.531990+0000","last_became_peered":"2023-11-24T06:25:49.531990+0000","last_unstale":"2023-11-24T06:25:50.563127+0000","last_undegraded":"2023-11-24T06:25:50.563127+0000","last_fullsized":"2023-11-24T06:25:50.563127+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:32.055977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527973+0000","last_change":"2023-11-24T06:25:50.550620+0000","last_active":"2023-11-24T06:25:51.527973+0000","last_peered":"2023-11-24T06:25:51.527973+0000","last_clean":"2023-11-24T06:25:51.527973+0000","last_became_active":"2023-11-24T06:25:50.550261+0000","last_became_peered":"2023-11-24T06:25:50.550261+0000","last_unstale":"2023-11-24T06:25:51.527973+0000","last_undegraded":"2023-11-24T06:25:51.527973+0000","last_fullsized":"2023-11-24T06:25:51.527973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:08.442324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.551350+0000","last_change":"2023-11-24T06:25:49.529122+0000","last_active":"2023-11-24T06:25:50.551350+0000","last_peered":"2023-11-24T06:25:50.551350+0000","last_clean":"2023-11-24T06:25:50.551350+0000","last_became_active":"2023-11-24T06:25:49.528854+0000","last_became_peered":"2023-11-24T06:25:49.528854+0000","last_unstale":"2023-11-24T06:25:50.551350+0000","last_undegraded":"2023-11-24T06:25:50.551350+0000","last_fullsized":"2023-11-24T06:25:50.551350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:38.725338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528188+0000","last_change":"2023-11-24T06:25:50.545691+0000","last_active":"2023-11-24T06:25:51.528188+0000","last_peered":"2023-11-24T06:25:51.528188+0000","last_clean":"2023-11-24T06:25:51.528188+0000","last_became_active":"2023-11-24T06:25:50.545404+0000","last_became_peered":"2023-11-24T06:25:50.545404+0000","last_unstale":"2023-11-24T06:25:51.528188+0000","last_undegraded":"2023-11-24T06:25:51.528188+0000","last_fullsized":"2023-11-24T06:25:51.528188+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.236990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.533043+0000","last_change":"2023-11-24T06:25:49.529495+0000","last_active":"2023-11-24T06:25:50.533043+0000","last_peered":"2023-11-24T06:25:50.533043+0000","last_clean":"2023-11-24T06:25:50.533043+0000","last_became_active":"2023-11-24T06:25:49.529256+0000","last_became_peered":"2023-11-24T06:25:49.529256+0000","last_unstale":"2023-11-24T06:25:50.533043+0000","last_undegraded":"2023-11-24T06:25:50.533043+0000","last_fullsized":"2023-11-24T06:25:50.533043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529226+0000","last_change":"2023-11-24T06:25:50.543251+0000","last_active":"2023-11-24T06:25:51.529226+0000","last_peered":"2023-11-24T06:25:51.529226+0000","last_clean":"2023-11-24T06:25:51.529226+0000","last_became_active":"2023-11-24T06:25:50.543000+0000","last_became_peered":"2023-11-24T06:25:50.543000+0000","last_unstale":"2023-11-24T06:25:51.529226+0000","last_undegraded":"2023-11-24T06:25:51.529226+0000","last_fullsized":"2023-11-24T06:25:51.529226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:07.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528783+0000","last_change":"2023-11-24T06:25:49.522891+0000","last_active":"2023-11-24T06:25:50.528783+0000","last_peered":"2023-11-24T06:25:50.528783+0000","last_clean":"2023-11-24T06:25:50.528783+0000","last_became_active":"2023-11-24T06:25:49.522605+0000","last_became_peered":"2023-11-24T06:25:49.522605+0000","last_unstale":"2023-11-24T06:25:50.528783+0000","last_undegraded":"2023-11-24T06:25:50.528783+0000","last_fullsized":"2023-11-24T06:25:50.528783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:41.758296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385336+0000","last_change":"2023-11-24T06:25:50.534096+0000","last_active":"2023-11-24T06:25:52.385336+0000","last_peered":"2023-11-24T06:25:52.385336+0000","last_clean":"2023-11-24T06:25:52.385336+0000","last_became_active":"2023-11-24T06:25:50.533802+0000","last_became_peered":"2023-11-24T06:25:50.533802+0000","last_unstale":"2023-11-24T06:25:52.385336+0000","last_undegraded":"2023-11-24T06:25:52.385336+0000","last_fullsized":"2023-11-24T06:25:52.385336+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:51.241710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527184+0000","last_change":"2023-11-24T06:25:49.522358+0000","last_active":"2023-11-24T06:25:50.527184+0000","last_peered":"2023-11-24T06:25:50.527184+0000","last_clean":"2023-11-24T06:25:50.527184+0000","last_became_active":"2023-11-24T06:25:49.521982+0000","last_became_peered":"2023-11-24T06:25:49.521982+0000","last_unstale":"2023-11-24T06:25:50.527184+0000","last_undegraded":"2023-11-24T06:25:50.527184+0000","last_fullsized":"2023-11-24T06:25:50.527184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529475+0000","last_change":"2023-11-24T06:25:50.538072+0000","last_active":"2023-11-24T06:25:51.529475+0000","last_peered":"2023-11-24T06:25:51.529475+0000","last_clean":"2023-11-24T06:25:51.529475+0000","last_became_active":"2023-11-24T06:25:50.537798+0000","last_became_peered":"2023-11-24T06:25:50.537798+0000","last_unstale":"2023-11-24T06:25:51.529475+0000","last_undegraded":"2023-11-24T06:25:51.529475+0000","last_fullsized":"2023-11-24T06:25:51.529475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:42.835952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.529344+0000","last_change":"2023-11-24T06:25:49.524636+0000","last_active":"2023-11-24T06:25:50.529344+0000","last_peered":"2023-11-24T06:25:50.529344+0000","last_clean":"2023-11-24T06:25:50.529344+0000","last_became_active":"2023-11-24T06:25:49.524351+0000","last_became_peered":"2023-11-24T06:25:49.524351+0000","last_unstale":"2023-11-24T06:25:50.529344+0000","last_undegraded":"2023-11-24T06:25:50.529344+0000","last_fullsized":"2023-11-24T06:25:50.529344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385529+0000","last_change":"2023-11-24T06:25:50.553681+0000","last_active":"2023-11-24T06:25:52.385529+0000","last_peered":"2023-11-24T06:25:52.385529+0000","last_clean":"2023-11-24T06:25:52.385529+0000","last_became_active":"2023-11-24T06:25:50.553357+0000","last_became_peered":"2023-11-24T06:25:50.553357+0000","last_unstale":"2023-11-24T06:25:52.385529+0000","last_undegraded":"2023-11-24T06:25:52.385529+0000","last_fullsized":"2023-11-24T06:25:52.385529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.445896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.554146+0000","last_change":"2023-11-24T06:25:49.525281+0000","last_active":"2023-11-24T06:25:50.554146+0000","last_peered":"2023-11-24T06:25:50.554146+0000","last_clean":"2023-11-24T06:25:50.554146+0000","last_became_active":"2023-11-24T06:25:49.524899+0000","last_became_peered":"2023-11-24T06:25:49.524899+0000","last_unstale":"2023-11-24T06:25:50.554146+0000","last_undegraded":"2023-11-24T06:25:50.554146+0000","last_fullsized":"2023-11-24T06:25:50.554146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:30.435820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528745+0000","last_change":"2023-11-24T06:25:50.548651+0000","last_active":"2023-11-24T06:25:51.528745+0000","last_peered":"2023-11-24T06:25:51.528745+0000","last_clean":"2023-11-24T06:25:51.528745+0000","last_became_active":"2023-11-24T06:25:50.548336+0000","last_became_peered":"2023-11-24T06:25:50.548336+0000","last_unstale":"2023-11-24T06:25:51.528745+0000","last_undegraded":"2023-11-24T06:25:51.528745+0000","last_fullsized":"2023-11-24T06:25:51.528745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:53:54.755693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.565479+0000","last_change":"2023-11-24T06:25:49.521264+0000","last_active":"2023-11-24T06:25:50.565479+0000","last_peered":"2023-11-24T06:25:50.565479+0000","last_clean":"2023-11-24T06:25:50.565479+0000","last_became_active":"2023-11-24T06:25:49.520985+0000","last_became_peered":"2023-11-24T06:25:49.520985+0000","last_unstale":"2023-11-24T06:25:50.565479+0000","last_undegraded":"2023-11-24T06:25:50.565479+0000","last_fullsized":"2023-11-24T06:25:50.565479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:48.462279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528854+0000","last_change":"2023-11-24T06:25:50.548573+0000","last_active":"2023-11-24T06:25:51.528854+0000","last_peered":"2023-11-24T06:25:51.528854+0000","last_clean":"2023-11-24T06:25:51.528854+0000","last_became_active":"2023-11-24T06:25:50.548129+0000","last_became_peered":"2023-11-24T06:25:50.548129+0000","last_unstale":"2023-11-24T06:25:51.528854+0000","last_undegraded":"2023-11-24T06:25:51.528854+0000","last_fullsized":"2023-11-24T06:25:51.528854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:39.732958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.572710+0000","last_change":"2023-11-24T06:25:49.531422+0000","last_active":"2023-11-24T06:25:50.572710+0000","last_peered":"2023-11-24T06:25:50.572710+0000","last_clean":"2023-11-24T06:25:50.572710+0000","last_became_active":"2023-11-24T06:25:49.531079+0000","last_became_peered":"2023-11-24T06:25:49.531079+0000","last_unstale":"2023-11-24T06:25:50.572710+0000","last_undegraded":"2023-11-24T06:25:50.572710+0000","last_fullsized":"2023-11-24T06:25:50.572710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:26.431372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528027+0000","last_change":"2023-11-24T06:25:50.536263+0000","last_active":"2023-11-24T06:25:51.528027+0000","last_peered":"2023-11-24T06:25:51.528027+0000","last_clean":"2023-11-24T06:25:51.528027+0000","last_became_active":"2023-11-24T06:25:50.535590+0000","last_became_peered":"2023-11-24T06:25:50.535590+0000","last_unstale":"2023-11-24T06:25:51.528027+0000","last_undegraded":"2023-11-24T06:25:51.528027+0000","last_fullsized":"2023-11-24T06:25:51.528027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:51.241710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527286+0000","last_change":"2023-11-24T06:25:49.531424+0000","last_active":"2023-11-24T06:25:50.527286+0000","last_peered":"2023-11-24T06:25:50.527286+0000","last_clean":"2023-11-24T06:25:50.527286+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:50.527286+0000","last_undegraded":"2023-11-24T06:25:50.527286+0000","last_fullsized":"2023-11-24T06:25:50.527286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:50.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385113+0000","last_change":"2023-11-24T06:25:50.554601+0000","last_active":"2023-11-24T06:25:52.385113+0000","last_peered":"2023-11-24T06:25:52.385113+0000","last_clean":"2023-11-24T06:25:52.385113+0000","last_became_active":"2023-11-24T06:25:50.554342+0000","last_became_peered":"2023-11-24T06:25:50.554342+0000","last_unstale":"2023-11-24T06:25:52.385113+0000","last_undegraded":"2023-11-24T06:25:52.385113+0000","last_fullsized":"2023-11-24T06:25:52.385113+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:24.373950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.539570+0000","last_change":"2023-11-24T06:25:49.531559+0000","last_active":"2023-11-24T06:25:50.539570+0000","last_peered":"2023-11-24T06:25:50.539570+0000","last_clean":"2023-11-24T06:25:50.539570+0000","last_became_active":"2023-11-24T06:25:49.531127+0000","last_became_peered":"2023-11-24T06:25:49.531127+0000","last_unstale":"2023-11-24T06:25:50.539570+0000","last_undegraded":"2023-11-24T06:25:50.539570+0000","last_fullsized":"2023-11-24T06:25:50.539570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:38.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529320+0000","last_change":"2023-11-24T06:25:50.544559+0000","last_active":"2023-11-24T06:25:51.529320+0000","last_peered":"2023-11-24T06:25:51.529320+0000","last_clean":"2023-11-24T06:25:51.529320+0000","last_became_active":"2023-11-24T06:25:50.544170+0000","last_became_peered":"2023-11-24T06:25:50.544170+0000","last_unstale":"2023-11-24T06:25:51.529320+0000","last_undegraded":"2023-11-24T06:25:51.529320+0000","last_fullsized":"2023-11-24T06:25:51.529320+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:43.722297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.556084+0000","last_change":"2023-11-24T06:25:49.529384+0000","last_active":"2023-11-24T06:25:50.556084+0000","last_peered":"2023-11-24T06:25:50.556084+0000","last_clean":"2023-11-24T06:25:50.556084+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:50.556084+0000","last_undegraded":"2023-11-24T06:25:50.556084+0000","last_fullsized":"2023-11-24T06:25:50.556084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:11.371497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903879+0000","last_change":"2023-11-24T06:25:50.549166+0000","last_active":"2023-11-24T06:25:51.903879+0000","last_peered":"2023-11-24T06:25:51.903879+0000","last_clean":"2023-11-24T06:25:51.903879+0000","last_became_active":"2023-11-24T06:25:50.548877+0000","last_became_peered":"2023-11-24T06:25:50.548877+0000","last_unstale":"2023-11-24T06:25:51.903879+0000","last_undegraded":"2023-11-24T06:25:51.903879+0000","last_fullsized":"2023-11-24T06:25:51.903879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:43.375134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563151+0000","last_change":"2023-11-24T06:25:49.533140+0000","last_active":"2023-11-24T06:25:50.563151+0000","last_peered":"2023-11-24T06:25:50.563151+0000","last_clean":"2023-11-24T06:25:50.563151+0000","last_became_active":"2023-11-24T06:25:49.532965+0000","last_became_peered":"2023-11-24T06:25:49.532965+0000","last_unstale":"2023-11-24T06:25:50.563151+0000","last_undegraded":"2023-11-24T06:25:50.563151+0000","last_fullsized":"2023-11-24T06:25:50.563151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:04.680380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528311+0000","last_change":"2023-11-24T06:25:50.557006+0000","last_active":"2023-11-24T06:25:51.528311+0000","last_peered":"2023-11-24T06:25:51.528311+0000","last_clean":"2023-11-24T06:25:51.528311+0000","last_became_active":"2023-11-24T06:25:50.556741+0000","last_became_peered":"2023-11-24T06:25:50.556741+0000","last_unstale":"2023-11-24T06:25:51.528311+0000","last_undegraded":"2023-11-24T06:25:51.528311+0000","last_fullsized":"2023-11-24T06:25:51.528311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:31.443439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.558413+0000","last_change":"2023-11-24T06:25:49.522374+0000","last_active":"2023-11-24T06:25:50.558413+0000","last_peered":"2023-11-24T06:25:50.558413+0000","last_clean":"2023-11-24T06:25:50.558413+0000","last_became_active":"2023-11-24T06:25:49.522053+0000","last_became_peered":"2023-11-24T06:25:49.522053+0000","last_unstale":"2023-11-24T06:25:50.558413+0000","last_undegraded":"2023-11-24T06:25:50.558413+0000","last_fullsized":"2023-11-24T06:25:50.558413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:28.289197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528555+0000","last_change":"2023-11-24T06:25:50.554685+0000","last_active":"2023-11-24T06:25:51.528555+0000","last_peered":"2023-11-24T06:25:51.528555+0000","last_clean":"2023-11-24T06:25:51.528555+0000","last_became_active":"2023-11-24T06:25:50.554379+0000","last_became_peered":"2023-11-24T06:25:50.554379+0000","last_unstale":"2023-11-24T06:25:51.528555+0000","last_undegraded":"2023-11-24T06:25:51.528555+0000","last_fullsized":"2023-11-24T06:25:51.528555+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:22.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.558291+0000","last_change":"2023-11-24T06:25:49.528898+0000","last_active":"2023-11-24T06:25:50.558291+0000","last_peered":"2023-11-24T06:25:50.558291+0000","last_clean":"2023-11-24T06:25:50.558291+0000","last_became_active":"2023-11-24T06:25:49.528642+0000","last_became_peered":"2023-11-24T06:25:49.528642+0000","last_unstale":"2023-11-24T06:25:50.558291+0000","last_undegraded":"2023-11-24T06:25:50.558291+0000","last_fullsized":"2023-11-24T06:25:50.558291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:42.438757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529524+0000","last_change":"2023-11-24T06:25:50.541711+0000","last_active":"2023-11-24T06:25:51.529524+0000","last_peered":"2023-11-24T06:25:51.529524+0000","last_clean":"2023-11-24T06:25:51.529524+0000","last_became_active":"2023-11-24T06:25:50.541407+0000","last_became_peered":"2023-11-24T06:25:50.541407+0000","last_unstale":"2023-11-24T06:25:51.529524+0000","last_undegraded":"2023-11-24T06:25:51.529524+0000","last_fullsized":"2023-11-24T06:25:51.529524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:10.604834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.540988+0000","last_change":"2023-11-24T06:25:49.531568+0000","last_active":"2023-11-24T06:25:50.540988+0000","last_peered":"2023-11-24T06:25:50.540988+0000","last_clean":"2023-11-24T06:25:50.540988+0000","last_became_active":"2023-11-24T06:25:49.531275+0000","last_became_peered":"2023-11-24T06:25:49.531275+0000","last_unstale":"2023-11-24T06:25:50.540988+0000","last_undegraded":"2023-11-24T06:25:50.540988+0000","last_fullsized":"2023-11-24T06:25:50.540988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:59.158530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385319+0000","last_change":"2023-11-24T06:25:50.553768+0000","last_active":"2023-11-24T06:25:52.385319+0000","last_peered":"2023-11-24T06:25:52.385319+0000","last_clean":"2023-11-24T06:25:52.385319+0000","last_became_active":"2023-11-24T06:25:50.553479+0000","last_became_peered":"2023-11-24T06:25:50.553479+0000","last_unstale":"2023-11-24T06:25:52.385319+0000","last_undegraded":"2023-11-24T06:25:52.385319+0000","last_fullsized":"2023-11-24T06:25:52.385319+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:59.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563066+0000","last_change":"2023-11-24T06:25:49.531979+0000","last_active":"2023-11-24T06:25:50.563066+0000","last_peered":"2023-11-24T06:25:50.563066+0000","last_clean":"2023-11-24T06:25:50.563066+0000","last_became_active":"2023-11-24T06:25:49.531699+0000","last_became_peered":"2023-11-24T06:25:49.531699+0000","last_unstale":"2023-11-24T06:25:50.563066+0000","last_undegraded":"2023-11-24T06:25:50.563066+0000","last_fullsized":"2023-11-24T06:25:50.563066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:51.611145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385893+0000","last_change":"2023-11-24T06:25:50.547085+0000","last_active":"2023-11-24T06:25:52.385893+0000","last_peered":"2023-11-24T06:25:52.385893+0000","last_clean":"2023-11-24T06:25:52.385893+0000","last_became_active":"2023-11-24T06:25:50.546821+0000","last_became_peered":"2023-11-24T06:25:50.546821+0000","last_unstale":"2023-11-24T06:25:52.385893+0000","last_undegraded":"2023-11-24T06:25:52.385893+0000","last_fullsized":"2023-11-24T06:25:52.385893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:10.604834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.533396+0000","last_change":"2023-11-24T06:25:49.528142+0000","last_active":"2023-11-24T06:25:50.533396+0000","last_peered":"2023-11-24T06:25:50.533396+0000","last_clean":"2023-11-24T06:25:50.533396+0000","last_became_active":"2023-11-24T06:25:49.527836+0000","last_became_peered":"2023-11-24T06:25:49.527836+0000","last_unstale":"2023-11-24T06:25:50.533396+0000","last_undegraded":"2023-11-24T06:25:50.533396+0000","last_fullsized":"2023-11-24T06:25:50.533396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903708+0000","last_change":"2023-11-24T06:25:50.543720+0000","last_active":"2023-11-24T06:25:51.903708+0000","last_peered":"2023-11-24T06:25:51.903708+0000","last_clean":"2023-11-24T06:25:51.903708+0000","last_became_active":"2023-11-24T06:25:50.543445+0000","last_became_peered":"2023-11-24T06:25:50.543445+0000","last_unstale":"2023-11-24T06:25:51.903708+0000","last_undegraded":"2023-11-24T06:25:51.903708+0000","last_fullsized":"2023-11-24T06:25:51.903708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:43.331737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.530249+0000","last_change":"2023-11-24T06:25:49.524448+0000","last_active":"2023-11-24T06:25:50.530249+0000","last_peered":"2023-11-24T06:25:50.530249+0000","last_clean":"2023-11-24T06:25:50.530249+0000","last_became_active":"2023-11-24T06:25:49.524159+0000","last_became_peered":"2023-11-24T06:25:49.524159+0000","last_unstale":"2023-11-24T06:25:50.530249+0000","last_undegraded":"2023-11-24T06:25:50.530249+0000","last_fullsized":"2023-11-24T06:25:50.530249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527814+0000","last_change":"2023-11-24T06:25:50.550108+0000","last_active":"2023-11-24T06:25:51.527814+0000","last_peered":"2023-11-24T06:25:51.527814+0000","last_clean":"2023-11-24T06:25:51.527814+0000","last_became_active":"2023-11-24T06:25:50.549838+0000","last_became_peered":"2023-11-24T06:25:50.549838+0000","last_unstale":"2023-11-24T06:25:51.527814+0000","last_undegraded":"2023-11-24T06:25:51.527814+0000","last_fullsized":"2023-11-24T06:25:51.527814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:14.915316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.526972+0000","last_change":"2023-11-24T06:25:49.531285+0000","last_active":"2023-11-24T06:25:50.526972+0000","last_peered":"2023-11-24T06:25:50.526972+0000","last_clean":"2023-11-24T06:25:50.526972+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:50.526972+0000","last_undegraded":"2023-11-24T06:25:50.526972+0000","last_fullsized":"2023-11-24T06:25:50.526972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528783+0000","last_change":"2023-11-24T06:25:50.548580+0000","last_active":"2023-11-24T06:25:51.528783+0000","last_peered":"2023-11-24T06:25:51.528783+0000","last_clean":"2023-11-24T06:25:51.528783+0000","last_became_active":"2023-11-24T06:25:50.548174+0000","last_became_peered":"2023-11-24T06:25:50.548174+0000","last_unstale":"2023-11-24T06:25:51.528783+0000","last_undegraded":"2023-11-24T06:25:51.528783+0000","last_fullsized":"2023-11-24T06:25:51.528783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:22.232490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.540695+0000","last_change":"2023-11-24T06:25:49.534058+0000","last_active":"2023-11-24T06:25:50.540695+0000","last_peered":"2023-11-24T06:25:50.540695+0000","last_clean":"2023-11-24T06:25:50.540695+0000","last_became_active":"2023-11-24T06:25:49.533728+0000","last_became_peered":"2023-11-24T06:25:49.533728+0000","last_unstale":"2023-11-24T06:25:50.540695+0000","last_undegraded":"2023-11-24T06:25:50.540695+0000","last_fullsized":"2023-11-24T06:25:50.540695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:41.828333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527993+0000","last_change":"2023-11-24T06:25:50.535053+0000","last_active":"2023-11-24T06:25:51.527993+0000","last_peered":"2023-11-24T06:25:51.527993+0000","last_clean":"2023-11-24T06:25:51.527993+0000","last_became_active":"2023-11-24T06:25:50.534759+0000","last_became_peered":"2023-11-24T06:25:50.534759+0000","last_unstale":"2023-11-24T06:25:51.527993+0000","last_undegraded":"2023-11-24T06:25:51.527993+0000","last_fullsized":"2023-11-24T06:25:51.527993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:06.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528873+0000","last_change":"2023-11-24T06:25:49.526982+0000","last_active":"2023-11-24T06:25:50.528873+0000","last_peered":"2023-11-24T06:25:50.528873+0000","last_clean":"2023-11-24T06:25:50.528873+0000","last_became_active":"2023-11-24T06:25:49.526688+0000","last_became_peered":"2023-11-24T06:25:49.526688+0000","last_unstale":"2023-11-24T06:25:50.528873+0000","last_undegraded":"2023-11-24T06:25:50.528873+0000","last_fullsized":"2023-11-24T06:25:50.528873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:43.762747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528792+0000","last_change":"2023-11-24T06:25:50.547067+0000","last_active":"2023-11-24T06:25:51.528792+0000","last_peered":"2023-11-24T06:25:51.528792+0000","last_clean":"2023-11-24T06:25:51.528792+0000","last_became_active":"2023-11-24T06:25:50.546738+0000","last_became_peered":"2023-11-24T06:25:50.546738+0000","last_unstale":"2023-11-24T06:25:51.528792+0000","last_undegraded":"2023-11-24T06:25:51.528792+0000","last_fullsized":"2023-11-24T06:25:51.528792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:39.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.536560+0000","last_change":"2023-11-24T06:25:49.529663+0000","last_active":"2023-11-24T06:25:50.536560+0000","last_peered":"2023-11-24T06:25:50.536560+0000","last_clean":"2023-11-24T06:25:50.536560+0000","last_became_active":"2023-11-24T06:25:49.529401+0000","last_became_peered":"2023-11-24T06:25:49.529401+0000","last_unstale":"2023-11-24T06:25:50.536560+0000","last_undegraded":"2023-11-24T06:25:50.536560+0000","last_fullsized":"2023-11-24T06:25:50.536560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:15.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385715+0000","last_change":"2023-11-24T06:25:50.557619+0000","last_active":"2023-11-24T06:25:52.385715+0000","last_peered":"2023-11-24T06:25:52.385715+0000","last_clean":"2023-11-24T06:25:52.385715+0000","last_became_active":"2023-11-24T06:25:50.557331+0000","last_became_peered":"2023-11-24T06:25:50.557331+0000","last_unstale":"2023-11-24T06:25:52.385715+0000","last_undegraded":"2023-11-24T06:25:52.385715+0000","last_fullsized":"2023-11-24T06:25:52.385715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:25.669514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527569+0000","last_change":"2023-11-24T06:25:49.525526+0000","last_active":"2023-11-24T06:25:50.527569+0000","last_peered":"2023-11-24T06:25:50.527569+0000","last_clean":"2023-11-24T06:25:50.527569+0000","last_became_active":"2023-11-24T06:25:49.525235+0000","last_became_peered":"2023-11-24T06:25:49.525235+0000","last_unstale":"2023-11-24T06:25:50.527569+0000","last_undegraded":"2023-11-24T06:25:50.527569+0000","last_fullsized":"2023-11-24T06:25:50.527569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:58.851781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385368+0000","last_change":"2023-11-24T06:25:50.557396+0000","last_active":"2023-11-24T06:25:52.385368+0000","last_peered":"2023-11-24T06:25:52.385368+0000","last_clean":"2023-11-24T06:25:52.385368+0000","last_became_active":"2023-11-24T06:25:50.557136+0000","last_became_peered":"2023-11-24T06:25:50.557136+0000","last_unstale":"2023-11-24T06:25:52.385368+0000","last_undegraded":"2023-11-24T06:25:52.385368+0000","last_fullsized":"2023-11-24T06:25:52.385368+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:01.760795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528433+0000","last_change":"2023-11-24T06:25:49.520031+0000","last_active":"2023-11-24T06:25:50.528433+0000","last_peered":"2023-11-24T06:25:50.528433+0000","last_clean":"2023-11-24T06:25:50.528433+0000","last_became_active":"2023-11-24T06:25:49.519789+0000","last_became_peered":"2023-11-24T06:25:49.519789+0000","last_unstale":"2023-11-24T06:25:50.528433+0000","last_undegraded":"2023-11-24T06:25:50.528433+0000","last_fullsized":"2023-11-24T06:25:50.528433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:15.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770644+0000","last_change":"2023-11-24T06:25:50.553590+0000","last_active":"2023-11-24T06:25:51.770644+0000","last_peered":"2023-11-24T06:25:51.770644+0000","last_clean":"2023-11-24T06:25:51.770644+0000","last_became_active":"2023-11-24T06:25:50.553224+0000","last_became_peered":"2023-11-24T06:25:50.553224+0000","last_unstale":"2023-11-24T06:25:51.770644+0000","last_undegraded":"2023-11-24T06:25:51.770644+0000","last_fullsized":"2023-11-24T06:25:51.770644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:31.443439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.536706+0000","last_change":"2023-11-24T06:25:49.525157+0000","last_active":"2023-11-24T06:25:50.536706+0000","last_peered":"2023-11-24T06:25:50.536706+0000","last_clean":"2023-11-24T06:25:50.536706+0000","last_became_active":"2023-11-24T06:25:49.524896+0000","last_became_peered":"2023-11-24T06:25:49.524896+0000","last_unstale":"2023-11-24T06:25:50.536706+0000","last_undegraded":"2023-11-24T06:25:50.536706+0000","last_fullsized":"2023-11-24T06:25:50.536706+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:12.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.384926+0000","last_change":"2023-11-24T06:25:50.547601+0000","last_active":"2023-11-24T06:25:52.384926+0000","last_peered":"2023-11-24T06:25:52.384926+0000","last_clean":"2023-11-24T06:25:52.384926+0000","last_became_active":"2023-11-24T06:25:50.547298+0000","last_became_peered":"2023-11-24T06:25:50.547298+0000","last_unstale":"2023-11-24T06:25:52.384926+0000","last_undegraded":"2023-11-24T06:25:52.384926+0000","last_fullsized":"2023-11-24T06:25:52.384926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:51.885765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528635+0000","last_change":"2023-11-24T06:25:49.528916+0000","last_active":"2023-11-24T06:25:50.528635+0000","last_peered":"2023-11-24T06:25:50.528635+0000","last_clean":"2023-11-24T06:25:50.528635+0000","last_became_active":"2023-11-24T06:25:49.528145+0000","last_became_peered":"2023-11-24T06:25:49.528145+0000","last_unstale":"2023-11-24T06:25:50.528635+0000","last_undegraded":"2023-11-24T06:25:50.528635+0000","last_fullsized":"2023-11-24T06:25:50.528635+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:36.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528149+0000","last_change":"2023-11-24T06:25:50.532640+0000","last_active":"2023-11-24T06:25:51.528149+0000","last_peered":"2023-11-24T06:25:51.528149+0000","last_clean":"2023-11-24T06:25:51.528149+0000","last_became_active":"2023-11-24T06:25:50.532381+0000","last_became_peered":"2023-11-24T06:25:50.532381+0000","last_unstale":"2023-11-24T06:25:51.528149+0000","last_undegraded":"2023-11-24T06:25:51.528149+0000","last_fullsized":"2023-11-24T06:25:51.528149+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:57.639964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563178+0000","last_change":"2023-11-24T06:25:49.533010+0000","last_active":"2023-11-24T06:25:50.563178+0000","last_peered":"2023-11-24T06:25:50.563178+0000","last_clean":"2023-11-24T06:25:50.563178+0000","last_became_active":"2023-11-24T06:25:49.532766+0000","last_became_peered":"2023-11-24T06:25:49.532766+0000","last_unstale":"2023-11-24T06:25:50.563178+0000","last_undegraded":"2023-11-24T06:25:50.563178+0000","last_fullsized":"2023-11-24T06:25:50.563178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:50.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527082+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:50.527082+0000","last_peered":"2023-11-24T06:25:50.527082+0000","last_clean":"2023-11-24T06:25:50.527082+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:50.527082+0000","last_undegraded":"2023-11-24T06:25:50.527082+0000","last_fullsized":"2023-11-24T06:25:50.527082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":804905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.501},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.39100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.308},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.499},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.377}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.436},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":804849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.253},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.373}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.622}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":216081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.77300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":215085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.314},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.377},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.621},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":216059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.316},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.625}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.373},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.36099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.438}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:56.504 INFO:tasks.ceph.mds.a.smithi114.stderr:2023-11-24T06:25:56.501+0000 7f5c1d82b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:25:59.203 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:25:59.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.335+0000 7fcf1b31c700 1 Processor -- start 2023-11-24T06:25:59.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.335+0000 7fcf1b31c700 1 -- start start 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf1b31c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf140b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf1b31c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcf1406c710 0x7fcf140b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf1b31c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf140af950 con 0x7fcf140adf00 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf1b31c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf14144a80 con 0x7fcf1406c710 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf1b31c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140afc30 con 0x7fcf141446a0 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf188b7700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcf140adf00 0x7fcf1406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56650/0 (socket says 172.21.15.114:56650) 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf188b7700 1 -- 172.21.15.114:0/3050971176 learned_addr learned my addr 172.21.15.114:0/3050971176 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:59.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897436026 0 0) 0x7fcf140af950 con 0x7fcf140adf00 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf000035b0 con 0x7fcf140adf00 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf190b8700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf140b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.336+0000 7fcf198b9700 1 --2- 172.21.15.114:0/3050971176 >> v2:172.21.15.172:3300/0 conn(0x7fcf1406c710 0x7fcf140b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550578544 0 0) 0x7fcf000035b0 con 0x7fcf140adf00 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 msgr2=0x7fcf140b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf140b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 >> v2:172.21.15.172:3300/0 conn(0x7fcf1406c710 msgr2=0x7fcf140b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/3050971176 >> v2:172.21.15.172:3300/0 conn(0x7fcf1406c710 0x7fcf140b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140aff10 con 0x7fcf140adf00 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fcf10001670 con 0x7fcf140adf00 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf198b9700 1 --2- 172.21.15.114:0/3050971176 >> v2:172.21.15.172:3300/0 conn(0x7fcf1406c710 0x7fcf140b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:59.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 >> v1:172.21.15.114:6789/0 conn(0x7fcf140adf00 legacy=0x7fcf1406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf00003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf00003f80 0x7fcf00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf00004d90 0x7fcf00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14144a80 con 0x7fcf00003f80 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140afc30 con 0x7fcf00004d90 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0000ba60 con 0x7fcf141446a0 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf00003f80 0x7fcf00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf190b8700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf00003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf198b9700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf00004d90 0x7fcf00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 -- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 msgr2=0x7fcf00003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf141446a0 0x7fcf00003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 -- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf00004d90 msgr2=0x7fcf00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf00004d90 0x7fcf00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 -- 172.21.15.114:0/3050971176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000035b0 con 0x7fcf00003f80 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf188b7700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf00003f80 0x7fcf00004850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c2f0 tx=0x7fcf00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8ae91e41746862 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000ce30 con 0x7fcf00003f80 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf10005960 con 0x7fcf00003f80 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.337+0000 7fcf0bfff700 1 -- 172.21.15.114:0/3050971176 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10005d10 con 0x7fcf00003f80 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 -- 172.21.15.114:0/3050971176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf00003f80 msgr2=0x7fcf00004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/3050971176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf00003f80 0x7fcf00004850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c2f0 tx=0x7fcf00005880 comp rx=0 tx=0).stop 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf198b9700 1 -- 172.21.15.114:0/3050971176 reap_dead start 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 -- 172.21.15.114:0/3050971176 shutdown_connections 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 -- 172.21.15.114:0/3050971176 >> 172.21.15.114:0/3050971176 conn(0x7fcf14071b90 msgr2=0x7fcf14071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:59.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 -- 172.21.15.114:0/3050971176 shutdown_connections 2023-11-24T06:25:59.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.338+0000 7fcf1b31c700 1 -- 172.21.15.114:0/3050971176 wait complete. 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.340+0000 7fcf1b31c700 1 Processor -- start 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.340+0000 7fcf1b31c700 1 -- start start 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf00003f80 0x7fcf1415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf1409bab0 0x7fcf14073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1409ee80 con 0x7fcf1409b380 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1409d600 con 0x7fcf1409bab0 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1409d8a0 con 0x7fcf00003f80 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40954/0 (socket says 172.21.15.114:40954) 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 -- 172.21.15.114:0/916584981 learned_addr learned my addr 172.21.15.114:0/916584981 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf190b8700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf00003f80 0x7fcf1415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf198b9700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf1409bab0 0x7fcf14073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 -- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf00003f80 msgr2=0x7fcf1415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf00003f80 0x7fcf1415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 -- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf1409bab0 msgr2=0x7fcf14073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf1409bab0 0x7fcf14073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 -- 172.21.15.114:0/916584981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1409db80 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf188b7700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf10000c00 tx=0x7fcf10001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10005ce0 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf10005e60 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1409de60 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.341+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10004ab0 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.342+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1409e420 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.343+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1409e5e0 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.343+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcf10004c10 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.343+0000 7fcf09ffb700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 0x7fcef806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.343+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcf1008abb0 con 0x7fcf1409b380 2023-11-24T06:25:59.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.343+0000 7fcf190b8700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 0x7fcef806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.344+0000 7fcf190b8700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 0x7fcef806d540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009b80 tx=0x7fcf04009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.348+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf10055af0 con 0x7fcf1409b380 2023-11-24T06:25:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.532+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf1409ee80 con 0x7fcef806b080 2023-11-24T06:25:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.540+0000 7fcf09ffb700 1 -- 172.21.15.114:0/916584981 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288760 (secure 0 0 0) 0x7fcf04019070 con 0x7fcef806b080 2023-11-24T06:25:59.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 msgr2=0x7fcef806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 0x7fcef806d540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009b80 tx=0x7fcf04009210 comp rx=0 tx=0).stop 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 msgr2=0x7fcf1409e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf10000c00 tx=0x7fcf10001a00 comp rx=0 tx=0).stop 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 shutdown_connections 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcef806b080 0x7fcef806d540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf1409bab0 0x7fcf14073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf00003f80 0x7fcf1415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 --2- 172.21.15.114:0/916584981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1409b380 0x7fcf1409e940 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 >> 172.21.15.114:0/916584981 conn(0x7fcf14071b90 msgr2=0x7fcf140b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 shutdown_connections 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.543+0000 7fcf1b31c700 1 -- 172.21.15.114:0/916584981 wait complete. 2023-11-24T06:25:59.550 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:25:59.558 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-11-24T06:25:58.863723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221540,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753180,"statfs":{"total":773094113280,"available":772867256320,"internally_reserved":0,"allocated":5480448,"data_stored":2901132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.336897"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529438+0000","last_change":"2023-11-24T06:25:50.531873+0000","last_active":"2023-11-24T06:25:51.529438+0000","last_peered":"2023-11-24T06:25:51.529438+0000","last_clean":"2023-11-24T06:25:51.529438+0000","last_became_active":"2023-11-24T06:25:50.531378+0000","last_became_peered":"2023-11-24T06:25:50.531378+0000","last_unstale":"2023-11-24T06:25:51.529438+0000","last_undegraded":"2023-11-24T06:25:51.529438+0000","last_fullsized":"2023-11-24T06:25:51.529438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:27.438992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770954+0000","last_change":"2023-11-24T06:25:50.546143+0000","last_active":"2023-11-24T06:25:51.770954+0000","last_peered":"2023-11-24T06:25:51.770954+0000","last_clean":"2023-11-24T06:25:51.770954+0000","last_became_active":"2023-11-24T06:25:50.545827+0000","last_became_peered":"2023-11-24T06:25:50.545827+0000","last_unstale":"2023-11-24T06:25:51.770954+0000","last_undegraded":"2023-11-24T06:25:51.770954+0000","last_fullsized":"2023-11-24T06:25:51.770954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:39.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385041+0000","last_change":"2023-11-24T06:25:50.546400+0000","last_active":"2023-11-24T06:25:52.385041+0000","last_peered":"2023-11-24T06:25:52.385041+0000","last_clean":"2023-11-24T06:25:52.385041+0000","last_became_active":"2023-11-24T06:25:50.546094+0000","last_became_peered":"2023-11-24T06:25:50.546094+0000","last_unstale":"2023-11-24T06:25:52.385041+0000","last_undegraded":"2023-11-24T06:25:52.385041+0000","last_fullsized":"2023-11-24T06:25:52.385041+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:14.915316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528123+0000","last_change":"2023-11-24T06:25:50.543369+0000","last_active":"2023-11-24T06:25:51.528123+0000","last_peered":"2023-11-24T06:25:51.528123+0000","last_clean":"2023-11-24T06:25:51.528123+0000","last_became_active":"2023-11-24T06:25:50.543047+0000","last_became_peered":"2023-11-24T06:25:50.543047+0000","last_unstale":"2023-11-24T06:25:51.528123+0000","last_undegraded":"2023-11-24T06:25:51.528123+0000","last_fullsized":"2023-11-24T06:25:51.528123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:59.859400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770776+0000","last_change":"2023-11-24T06:25:50.545946+0000","last_active":"2023-11-24T06:25:51.770776+0000","last_peered":"2023-11-24T06:25:51.770776+0000","last_clean":"2023-11-24T06:25:51.770776+0000","last_became_active":"2023-11-24T06:25:50.545667+0000","last_became_peered":"2023-11-24T06:25:50.545667+0000","last_unstale":"2023-11-24T06:25:51.770776+0000","last_undegraded":"2023-11-24T06:25:51.770776+0000","last_fullsized":"2023-11-24T06:25:51.770776+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:44.770366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385273+0000","last_change":"2023-11-24T06:25:50.546095+0000","last_active":"2023-11-24T06:25:52.385273+0000","last_peered":"2023-11-24T06:25:52.385273+0000","last_clean":"2023-11-24T06:25:52.385273+0000","last_became_active":"2023-11-24T06:25:50.545799+0000","last_became_peered":"2023-11-24T06:25:50.545799+0000","last_unstale":"2023-11-24T06:25:52.385273+0000","last_undegraded":"2023-11-24T06:25:52.385273+0000","last_fullsized":"2023-11-24T06:25:52.385273+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:00.259511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528438+0000","last_change":"2023-11-24T06:25:50.548753+0000","last_active":"2023-11-24T06:25:51.528438+0000","last_peered":"2023-11-24T06:25:51.528438+0000","last_clean":"2023-11-24T06:25:51.528438+0000","last_became_active":"2023-11-24T06:25:50.548478+0000","last_became_peered":"2023-11-24T06:25:50.548478+0000","last_unstale":"2023-11-24T06:25:51.528438+0000","last_undegraded":"2023-11-24T06:25:51.528438+0000","last_fullsized":"2023-11-24T06:25:51.528438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:41.853651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528221+0000","last_change":"2023-11-24T06:25:50.544689+0000","last_active":"2023-11-24T06:25:51.528221+0000","last_peered":"2023-11-24T06:25:51.528221+0000","last_clean":"2023-11-24T06:25:51.528221+0000","last_became_active":"2023-11-24T06:25:50.544431+0000","last_became_peered":"2023-11-24T06:25:50.544431+0000","last_unstale":"2023-11-24T06:25:51.528221+0000","last_undegraded":"2023-11-24T06:25:51.528221+0000","last_fullsized":"2023-11-24T06:25:51.528221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:43.446376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528194+0000","last_change":"2023-11-24T06:25:50.550332+0000","last_active":"2023-11-24T06:25:51.528194+0000","last_peered":"2023-11-24T06:25:51.528194+0000","last_clean":"2023-11-24T06:25:51.528194+0000","last_became_active":"2023-11-24T06:25:50.548627+0000","last_became_peered":"2023-11-24T06:25:50.548627+0000","last_unstale":"2023-11-24T06:25:51.528194+0000","last_undegraded":"2023-11-24T06:25:51.528194+0000","last_fullsized":"2023-11-24T06:25:51.528194+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:35.110793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528193+0000","last_change":"2023-11-24T06:25:50.543547+0000","last_active":"2023-11-24T06:25:51.528193+0000","last_peered":"2023-11-24T06:25:51.528193+0000","last_clean":"2023-11-24T06:25:51.528193+0000","last_became_active":"2023-11-24T06:25:50.543257+0000","last_became_peered":"2023-11-24T06:25:50.543257+0000","last_unstale":"2023-11-24T06:25:51.528193+0000","last_undegraded":"2023-11-24T06:25:51.528193+0000","last_fullsized":"2023-11-24T06:25:51.528193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:51.509785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528303+0000","last_change":"2023-11-24T06:25:50.554529+0000","last_active":"2023-11-24T06:25:51.528303+0000","last_peered":"2023-11-24T06:25:51.528303+0000","last_clean":"2023-11-24T06:25:51.528303+0000","last_became_active":"2023-11-24T06:25:50.554245+0000","last_became_peered":"2023-11-24T06:25:50.554245+0000","last_unstale":"2023-11-24T06:25:51.528303+0000","last_undegraded":"2023-11-24T06:25:51.528303+0000","last_fullsized":"2023-11-24T06:25:51.528303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:44.770366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770851+0000","last_change":"2023-11-24T06:25:50.545580+0000","last_active":"2023-11-24T06:25:51.770851+0000","last_peered":"2023-11-24T06:25:51.770851+0000","last_clean":"2023-11-24T06:25:51.770851+0000","last_became_active":"2023-11-24T06:25:50.545280+0000","last_became_peered":"2023-11-24T06:25:50.545280+0000","last_unstale":"2023-11-24T06:25:51.770851+0000","last_undegraded":"2023-11-24T06:25:51.770851+0000","last_fullsized":"2023-11-24T06:25:51.770851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:41.853651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903795+0000","last_change":"2023-11-24T06:25:50.558908+0000","last_active":"2023-11-24T06:25:51.903795+0000","last_peered":"2023-11-24T06:25:51.903795+0000","last_clean":"2023-11-24T06:25:51.903795+0000","last_became_active":"2023-11-24T06:25:50.558628+0000","last_became_peered":"2023-11-24T06:25:50.558628+0000","last_unstale":"2023-11-24T06:25:51.903795+0000","last_undegraded":"2023-11-24T06:25:51.903795+0000","last_fullsized":"2023-11-24T06:25:51.903795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:19.029120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527910+0000","last_change":"2023-11-24T06:25:50.554035+0000","last_active":"2023-11-24T06:25:51.527910+0000","last_peered":"2023-11-24T06:25:51.527910+0000","last_clean":"2023-11-24T06:25:51.527910+0000","last_became_active":"2023-11-24T06:25:50.553744+0000","last_became_peered":"2023-11-24T06:25:50.553744+0000","last_unstale":"2023-11-24T06:25:51.527910+0000","last_undegraded":"2023-11-24T06:25:51.527910+0000","last_fullsized":"2023-11-24T06:25:51.527910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:44.191847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385128+0000","last_change":"2023-11-24T06:25:50.547859+0000","last_active":"2023-11-24T06:25:52.385128+0000","last_peered":"2023-11-24T06:25:52.385128+0000","last_clean":"2023-11-24T06:25:52.385128+0000","last_became_active":"2023-11-24T06:25:50.546438+0000","last_became_peered":"2023-11-24T06:25:50.546438+0000","last_unstale":"2023-11-24T06:25:52.385128+0000","last_undegraded":"2023-11-24T06:25:52.385128+0000","last_fullsized":"2023-11-24T06:25:52.385128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:51.032214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528282+0000","last_change":"2023-11-24T06:25:50.543140+0000","last_active":"2023-11-24T06:25:51.528282+0000","last_peered":"2023-11-24T06:25:51.528282+0000","last_clean":"2023-11-24T06:25:51.528282+0000","last_became_active":"2023-11-24T06:25:50.542818+0000","last_became_peered":"2023-11-24T06:25:50.542818+0000","last_unstale":"2023-11-24T06:25:51.528282+0000","last_undegraded":"2023-11-24T06:25:51.528282+0000","last_fullsized":"2023-11-24T06:25:51.528282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.373959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385134+0000","last_change":"2023-11-24T06:25:50.539395+0000","last_active":"2023-11-24T06:25:52.385134+0000","last_peered":"2023-11-24T06:25:52.385134+0000","last_clean":"2023-11-24T06:25:52.385134+0000","last_became_active":"2023-11-24T06:25:50.539030+0000","last_became_peered":"2023-11-24T06:25:50.539030+0000","last_unstale":"2023-11-24T06:25:52.385134+0000","last_undegraded":"2023-11-24T06:25:52.385134+0000","last_fullsized":"2023-11-24T06:25:52.385134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:08.442324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385190+0000","last_change":"2023-11-24T06:25:50.534513+0000","last_active":"2023-11-24T06:25:52.385190+0000","last_peered":"2023-11-24T06:25:52.385190+0000","last_clean":"2023-11-24T06:25:52.385190+0000","last_became_active":"2023-11-24T06:25:50.534187+0000","last_became_peered":"2023-11-24T06:25:50.534187+0000","last_unstale":"2023-11-24T06:25:52.385190+0000","last_undegraded":"2023-11-24T06:25:52.385190+0000","last_fullsized":"2023-11-24T06:25:52.385190+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:06.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529501+0000","last_change":"2023-11-24T06:25:50.543322+0000","last_active":"2023-11-24T06:25:51.529501+0000","last_peered":"2023-11-24T06:25:51.529501+0000","last_clean":"2023-11-24T06:25:51.529501+0000","last_became_active":"2023-11-24T06:25:50.543067+0000","last_became_peered":"2023-11-24T06:25:50.543067+0000","last_unstale":"2023-11-24T06:25:51.529501+0000","last_undegraded":"2023-11-24T06:25:51.529501+0000","last_fullsized":"2023-11-24T06:25:51.529501+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:29.296816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903990+0000","last_change":"2023-11-24T06:25:50.548979+0000","last_active":"2023-11-24T06:25:51.903990+0000","last_peered":"2023-11-24T06:25:51.903990+0000","last_clean":"2023-11-24T06:25:51.903990+0000","last_became_active":"2023-11-24T06:25:50.548725+0000","last_became_peered":"2023-11-24T06:25:50.548725+0000","last_unstale":"2023-11-24T06:25:51.903990+0000","last_undegraded":"2023-11-24T06:25:51.903990+0000","last_fullsized":"2023-11-24T06:25:51.903990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:12.379116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385507+0000","last_change":"2023-11-24T06:25:50.545765+0000","last_active":"2023-11-24T06:25:52.385507+0000","last_peered":"2023-11-24T06:25:52.385507+0000","last_clean":"2023-11-24T06:25:52.385507+0000","last_became_active":"2023-11-24T06:25:50.545446+0000","last_became_peered":"2023-11-24T06:25:50.545446+0000","last_unstale":"2023-11-24T06:25:52.385507+0000","last_undegraded":"2023-11-24T06:25:52.385507+0000","last_fullsized":"2023-11-24T06:25:52.385507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:56.447309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528065+0000","last_change":"2023-11-24T06:25:50.545609+0000","last_active":"2023-11-24T06:25:51.528065+0000","last_peered":"2023-11-24T06:25:51.528065+0000","last_clean":"2023-11-24T06:25:51.528065+0000","last_became_active":"2023-11-24T06:25:50.544865+0000","last_became_peered":"2023-11-24T06:25:50.544865+0000","last_unstale":"2023-11-24T06:25:51.528065+0000","last_undegraded":"2023-11-24T06:25:51.528065+0000","last_fullsized":"2023-11-24T06:25:51.528065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:42.713204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529210+0000","last_change":"2023-11-24T06:25:50.544181+0000","last_active":"2023-11-24T06:25:51.529210+0000","last_peered":"2023-11-24T06:25:51.529210+0000","last_clean":"2023-11-24T06:25:51.529210+0000","last_became_active":"2023-11-24T06:25:50.543881+0000","last_became_peered":"2023-11-24T06:25:50.543881+0000","last_unstale":"2023-11-24T06:25:51.529210+0000","last_undegraded":"2023-11-24T06:25:51.529210+0000","last_fullsized":"2023-11-24T06:25:51.529210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:13.713094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528681+0000","last_change":"2023-11-24T06:25:50.540318+0000","last_active":"2023-11-24T06:25:51.528681+0000","last_peered":"2023-11-24T06:25:51.528681+0000","last_clean":"2023-11-24T06:25:51.528681+0000","last_became_active":"2023-11-24T06:25:50.540040+0000","last_became_peered":"2023-11-24T06:25:50.540040+0000","last_unstale":"2023-11-24T06:25:51.528681+0000","last_undegraded":"2023-11-24T06:25:51.528681+0000","last_fullsized":"2023-11-24T06:25:51.528681+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:58.401801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529012+0000","last_change":"2023-11-24T06:25:50.558884+0000","last_active":"2023-11-24T06:25:51.529012+0000","last_peered":"2023-11-24T06:25:51.529012+0000","last_clean":"2023-11-24T06:25:51.529012+0000","last_became_active":"2023-11-24T06:25:50.558620+0000","last_became_peered":"2023-11-24T06:25:50.558620+0000","last_unstale":"2023-11-24T06:25:51.529012+0000","last_undegraded":"2023-11-24T06:25:51.529012+0000","last_fullsized":"2023-11-24T06:25:51.529012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:16.475409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903866+0000","last_change":"2023-11-24T06:25:50.555376+0000","last_active":"2023-11-24T06:25:51.903866+0000","last_peered":"2023-11-24T06:25:51.903866+0000","last_clean":"2023-11-24T06:25:51.903866+0000","last_became_active":"2023-11-24T06:25:50.555137+0000","last_became_peered":"2023-11-24T06:25:50.555137+0000","last_unstale":"2023-11-24T06:25:51.903866+0000","last_undegraded":"2023-11-24T06:25:51.903866+0000","last_fullsized":"2023-11-24T06:25:51.903866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:33.092309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385044+0000","last_change":"2023-11-24T06:25:50.540012+0000","last_active":"2023-11-24T06:25:52.385044+0000","last_peered":"2023-11-24T06:25:52.385044+0000","last_clean":"2023-11-24T06:25:52.385044+0000","last_became_active":"2023-11-24T06:25:50.539757+0000","last_became_peered":"2023-11-24T06:25:50.539757+0000","last_unstale":"2023-11-24T06:25:52.385044+0000","last_undegraded":"2023-11-24T06:25:52.385044+0000","last_fullsized":"2023-11-24T06:25:52.385044+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:08.477759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.771040+0000","last_change":"2023-11-24T06:25:50.545665+0000","last_active":"2023-11-24T06:25:51.771040+0000","last_peered":"2023-11-24T06:25:51.771040+0000","last_clean":"2023-11-24T06:25:51.771040+0000","last_became_active":"2023-11-24T06:25:50.545297+0000","last_became_peered":"2023-11-24T06:25:50.545297+0000","last_unstale":"2023-11-24T06:25:51.771040+0000","last_undegraded":"2023-11-24T06:25:51.771040+0000","last_fullsized":"2023-11-24T06:25:51.771040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:43.722297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528256+0000","last_change":"2023-11-24T06:25:50.553740+0000","last_active":"2023-11-24T06:25:51.528256+0000","last_peered":"2023-11-24T06:25:51.528256+0000","last_clean":"2023-11-24T06:25:51.528256+0000","last_became_active":"2023-11-24T06:25:50.553475+0000","last_became_peered":"2023-11-24T06:25:50.553475+0000","last_unstale":"2023-11-24T06:25:51.528256+0000","last_undegraded":"2023-11-24T06:25:51.528256+0000","last_fullsized":"2023-11-24T06:25:51.528256+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:44.060556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385111+0000","last_change":"2023-11-24T06:25:50.531496+0000","last_active":"2023-11-24T06:25:52.385111+0000","last_peered":"2023-11-24T06:25:52.385111+0000","last_clean":"2023-11-24T06:25:52.385111+0000","last_became_active":"2023-11-24T06:25:50.531201+0000","last_became_peered":"2023-11-24T06:25:50.531201+0000","last_unstale":"2023-11-24T06:25:52.385111+0000","last_undegraded":"2023-11-24T06:25:52.385111+0000","last_fullsized":"2023-11-24T06:25:52.385111+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:44.191847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385322+0000","last_change":"2023-11-24T06:25:50.548666+0000","last_active":"2023-11-24T06:25:52.385322+0000","last_peered":"2023-11-24T06:25:52.385322+0000","last_clean":"2023-11-24T06:25:52.385322+0000","last_became_active":"2023-11-24T06:25:50.548138+0000","last_became_peered":"2023-11-24T06:25:50.548138+0000","last_unstale":"2023-11-24T06:25:52.385322+0000","last_undegraded":"2023-11-24T06:25:52.385322+0000","last_fullsized":"2023-11-24T06:25:52.385322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:45.473450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903738+0000","last_change":"2023-11-24T06:25:50.556726+0000","last_active":"2023-11-24T06:25:51.903738+0000","last_peered":"2023-11-24T06:25:51.903738+0000","last_clean":"2023-11-24T06:25:51.903738+0000","last_became_active":"2023-11-24T06:25:50.556409+0000","last_became_peered":"2023-11-24T06:25:50.556409+0000","last_unstale":"2023-11-24T06:25:51.903738+0000","last_undegraded":"2023-11-24T06:25:51.903738+0000","last_fullsized":"2023-11-24T06:25:51.903738+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:55.903587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385988+0000","last_change":"2023-11-24T06:25:50.552279+0000","last_active":"2023-11-24T06:25:52.385988+0000","last_peered":"2023-11-24T06:25:52.385988+0000","last_clean":"2023-11-24T06:25:52.385988+0000","last_became_active":"2023-11-24T06:25:50.552073+0000","last_became_peered":"2023-11-24T06:25:50.552073+0000","last_unstale":"2023-11-24T06:25:52.385988+0000","last_undegraded":"2023-11-24T06:25:52.385988+0000","last_fullsized":"2023-11-24T06:25:52.385988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:51.885765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563196+0000","last_change":"2023-11-24T06:25:49.531680+0000","last_active":"2023-11-24T06:25:50.563196+0000","last_peered":"2023-11-24T06:25:50.563196+0000","last_clean":"2023-11-24T06:25:50.563196+0000","last_became_active":"2023-11-24T06:25:49.531361+0000","last_became_peered":"2023-11-24T06:25:49.531361+0000","last_unstale":"2023-11-24T06:25:50.563196+0000","last_undegraded":"2023-11-24T06:25:50.563196+0000","last_fullsized":"2023-11-24T06:25:50.563196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:42.714678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528598+0000","last_change":"2023-11-24T06:25:50.542384+0000","last_active":"2023-11-24T06:25:51.528598+0000","last_peered":"2023-11-24T06:25:51.528598+0000","last_clean":"2023-11-24T06:25:51.528598+0000","last_became_active":"2023-11-24T06:25:50.542079+0000","last_became_peered":"2023-11-24T06:25:50.542079+0000","last_unstale":"2023-11-24T06:25:51.528598+0000","last_undegraded":"2023-11-24T06:25:51.528598+0000","last_fullsized":"2023-11-24T06:25:51.528598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:53.015676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.550312+0000","last_change":"2023-11-24T06:25:49.525389+0000","last_active":"2023-11-24T06:25:50.550312+0000","last_peered":"2023-11-24T06:25:50.550312+0000","last_clean":"2023-11-24T06:25:50.550312+0000","last_became_active":"2023-11-24T06:25:49.524856+0000","last_became_peered":"2023-11-24T06:25:49.524856+0000","last_unstale":"2023-11-24T06:25:50.550312+0000","last_undegraded":"2023-11-24T06:25:50.550312+0000","last_fullsized":"2023-11-24T06:25:50.550312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770935+0000","last_change":"2023-11-24T06:25:50.554666+0000","last_active":"2023-11-24T06:25:51.770935+0000","last_peered":"2023-11-24T06:25:51.770935+0000","last_clean":"2023-11-24T06:25:51.770935+0000","last_became_active":"2023-11-24T06:25:50.554392+0000","last_became_peered":"2023-11-24T06:25:50.554392+0000","last_unstale":"2023-11-24T06:25:51.770935+0000","last_undegraded":"2023-11-24T06:25:51.770935+0000","last_fullsized":"2023-11-24T06:25:51.770935+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:53:54.755693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.560598+0000","last_change":"2023-11-24T06:25:49.530053+0000","last_active":"2023-11-24T06:25:50.560598+0000","last_peered":"2023-11-24T06:25:50.560598+0000","last_clean":"2023-11-24T06:25:50.560598+0000","last_became_active":"2023-11-24T06:25:49.529544+0000","last_became_peered":"2023-11-24T06:25:49.529544+0000","last_unstale":"2023-11-24T06:25:50.560598+0000","last_undegraded":"2023-11-24T06:25:50.560598+0000","last_fullsized":"2023-11-24T06:25:50.560598+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:06.466292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385980+0000","last_change":"2023-11-24T06:25:50.552742+0000","last_active":"2023-11-24T06:25:52.385980+0000","last_peered":"2023-11-24T06:25:52.385980+0000","last_clean":"2023-11-24T06:25:52.385980+0000","last_became_active":"2023-11-24T06:25:50.549874+0000","last_became_peered":"2023-11-24T06:25:50.549874+0000","last_unstale":"2023-11-24T06:25:52.385980+0000","last_undegraded":"2023-11-24T06:25:52.385980+0000","last_fullsized":"2023-11-24T06:25:52.385980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:58.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.553637+0000","last_change":"2023-11-24T06:25:49.529413+0000","last_active":"2023-11-24T06:25:50.553637+0000","last_peered":"2023-11-24T06:25:50.553637+0000","last_clean":"2023-11-24T06:25:50.553637+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:50.553637+0000","last_undegraded":"2023-11-24T06:25:50.553637+0000","last_fullsized":"2023-11-24T06:25:50.553637+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.384929+0000","last_change":"2023-11-24T06:25:50.544430+0000","last_active":"2023-11-24T06:25:52.384929+0000","last_peered":"2023-11-24T06:25:52.384929+0000","last_clean":"2023-11-24T06:25:52.384929+0000","last_became_active":"2023-11-24T06:25:50.544143+0000","last_became_peered":"2023-11-24T06:25:50.544143+0000","last_unstale":"2023-11-24T06:25:52.384929+0000","last_undegraded":"2023-11-24T06:25:52.384929+0000","last_fullsized":"2023-11-24T06:25:52.384929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:59:58.732630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563127+0000","last_change":"2023-11-24T06:25:49.532801+0000","last_active":"2023-11-24T06:25:50.563127+0000","last_peered":"2023-11-24T06:25:50.563127+0000","last_clean":"2023-11-24T06:25:50.563127+0000","last_became_active":"2023-11-24T06:25:49.531990+0000","last_became_peered":"2023-11-24T06:25:49.531990+0000","last_unstale":"2023-11-24T06:25:50.563127+0000","last_undegraded":"2023-11-24T06:25:50.563127+0000","last_fullsized":"2023-11-24T06:25:50.563127+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:32.055977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527973+0000","last_change":"2023-11-24T06:25:50.550620+0000","last_active":"2023-11-24T06:25:51.527973+0000","last_peered":"2023-11-24T06:25:51.527973+0000","last_clean":"2023-11-24T06:25:51.527973+0000","last_became_active":"2023-11-24T06:25:50.550261+0000","last_became_peered":"2023-11-24T06:25:50.550261+0000","last_unstale":"2023-11-24T06:25:51.527973+0000","last_undegraded":"2023-11-24T06:25:51.527973+0000","last_fullsized":"2023-11-24T06:25:51.527973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:08.442324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.551350+0000","last_change":"2023-11-24T06:25:49.529122+0000","last_active":"2023-11-24T06:25:50.551350+0000","last_peered":"2023-11-24T06:25:50.551350+0000","last_clean":"2023-11-24T06:25:50.551350+0000","last_became_active":"2023-11-24T06:25:49.528854+0000","last_became_peered":"2023-11-24T06:25:49.528854+0000","last_unstale":"2023-11-24T06:25:50.551350+0000","last_undegraded":"2023-11-24T06:25:50.551350+0000","last_fullsized":"2023-11-24T06:25:50.551350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:38.725338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528188+0000","last_change":"2023-11-24T06:25:50.545691+0000","last_active":"2023-11-24T06:25:51.528188+0000","last_peered":"2023-11-24T06:25:51.528188+0000","last_clean":"2023-11-24T06:25:51.528188+0000","last_became_active":"2023-11-24T06:25:50.545404+0000","last_became_peered":"2023-11-24T06:25:50.545404+0000","last_unstale":"2023-11-24T06:25:51.528188+0000","last_undegraded":"2023-11-24T06:25:51.528188+0000","last_fullsized":"2023-11-24T06:25:51.528188+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.236990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.533043+0000","last_change":"2023-11-24T06:25:49.529495+0000","last_active":"2023-11-24T06:25:50.533043+0000","last_peered":"2023-11-24T06:25:50.533043+0000","last_clean":"2023-11-24T06:25:50.533043+0000","last_became_active":"2023-11-24T06:25:49.529256+0000","last_became_peered":"2023-11-24T06:25:49.529256+0000","last_unstale":"2023-11-24T06:25:50.533043+0000","last_undegraded":"2023-11-24T06:25:50.533043+0000","last_fullsized":"2023-11-24T06:25:50.533043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529226+0000","last_change":"2023-11-24T06:25:50.543251+0000","last_active":"2023-11-24T06:25:51.529226+0000","last_peered":"2023-11-24T06:25:51.529226+0000","last_clean":"2023-11-24T06:25:51.529226+0000","last_became_active":"2023-11-24T06:25:50.543000+0000","last_became_peered":"2023-11-24T06:25:50.543000+0000","last_unstale":"2023-11-24T06:25:51.529226+0000","last_undegraded":"2023-11-24T06:25:51.529226+0000","last_fullsized":"2023-11-24T06:25:51.529226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:17:07.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528783+0000","last_change":"2023-11-24T06:25:49.522891+0000","last_active":"2023-11-24T06:25:50.528783+0000","last_peered":"2023-11-24T06:25:50.528783+0000","last_clean":"2023-11-24T06:25:50.528783+0000","last_became_active":"2023-11-24T06:25:49.522605+0000","last_became_peered":"2023-11-24T06:25:49.522605+0000","last_unstale":"2023-11-24T06:25:50.528783+0000","last_undegraded":"2023-11-24T06:25:50.528783+0000","last_fullsized":"2023-11-24T06:25:50.528783+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:41.758296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385336+0000","last_change":"2023-11-24T06:25:50.534096+0000","last_active":"2023-11-24T06:25:52.385336+0000","last_peered":"2023-11-24T06:25:52.385336+0000","last_clean":"2023-11-24T06:25:52.385336+0000","last_became_active":"2023-11-24T06:25:50.533802+0000","last_became_peered":"2023-11-24T06:25:50.533802+0000","last_unstale":"2023-11-24T06:25:52.385336+0000","last_undegraded":"2023-11-24T06:25:52.385336+0000","last_fullsized":"2023-11-24T06:25:52.385336+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:51.241710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527184+0000","last_change":"2023-11-24T06:25:49.522358+0000","last_active":"2023-11-24T06:25:50.527184+0000","last_peered":"2023-11-24T06:25:50.527184+0000","last_clean":"2023-11-24T06:25:50.527184+0000","last_became_active":"2023-11-24T06:25:49.521982+0000","last_became_peered":"2023-11-24T06:25:49.521982+0000","last_unstale":"2023-11-24T06:25:50.527184+0000","last_undegraded":"2023-11-24T06:25:50.527184+0000","last_fullsized":"2023-11-24T06:25:50.527184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529475+0000","last_change":"2023-11-24T06:25:50.538072+0000","last_active":"2023-11-24T06:25:51.529475+0000","last_peered":"2023-11-24T06:25:51.529475+0000","last_clean":"2023-11-24T06:25:51.529475+0000","last_became_active":"2023-11-24T06:25:50.537798+0000","last_became_peered":"2023-11-24T06:25:50.537798+0000","last_unstale":"2023-11-24T06:25:51.529475+0000","last_undegraded":"2023-11-24T06:25:51.529475+0000","last_fullsized":"2023-11-24T06:25:51.529475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:42.835952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.529344+0000","last_change":"2023-11-24T06:25:49.524636+0000","last_active":"2023-11-24T06:25:50.529344+0000","last_peered":"2023-11-24T06:25:50.529344+0000","last_clean":"2023-11-24T06:25:50.529344+0000","last_became_active":"2023-11-24T06:25:49.524351+0000","last_became_peered":"2023-11-24T06:25:49.524351+0000","last_unstale":"2023-11-24T06:25:50.529344+0000","last_undegraded":"2023-11-24T06:25:50.529344+0000","last_fullsized":"2023-11-24T06:25:50.529344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.366340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385529+0000","last_change":"2023-11-24T06:25:50.553681+0000","last_active":"2023-11-24T06:25:52.385529+0000","last_peered":"2023-11-24T06:25:52.385529+0000","last_clean":"2023-11-24T06:25:52.385529+0000","last_became_active":"2023-11-24T06:25:50.553357+0000","last_became_peered":"2023-11-24T06:25:50.553357+0000","last_unstale":"2023-11-24T06:25:52.385529+0000","last_undegraded":"2023-11-24T06:25:52.385529+0000","last_fullsized":"2023-11-24T06:25:52.385529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:44:04.445896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.554146+0000","last_change":"2023-11-24T06:25:49.525281+0000","last_active":"2023-11-24T06:25:50.554146+0000","last_peered":"2023-11-24T06:25:50.554146+0000","last_clean":"2023-11-24T06:25:50.554146+0000","last_became_active":"2023-11-24T06:25:49.524899+0000","last_became_peered":"2023-11-24T06:25:49.524899+0000","last_unstale":"2023-11-24T06:25:50.554146+0000","last_undegraded":"2023-11-24T06:25:50.554146+0000","last_fullsized":"2023-11-24T06:25:50.554146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:30.435820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528745+0000","last_change":"2023-11-24T06:25:50.548651+0000","last_active":"2023-11-24T06:25:51.528745+0000","last_peered":"2023-11-24T06:25:51.528745+0000","last_clean":"2023-11-24T06:25:51.528745+0000","last_became_active":"2023-11-24T06:25:50.548336+0000","last_became_peered":"2023-11-24T06:25:50.548336+0000","last_unstale":"2023-11-24T06:25:51.528745+0000","last_undegraded":"2023-11-24T06:25:51.528745+0000","last_fullsized":"2023-11-24T06:25:51.528745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:53:54.755693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.565479+0000","last_change":"2023-11-24T06:25:49.521264+0000","last_active":"2023-11-24T06:25:50.565479+0000","last_peered":"2023-11-24T06:25:50.565479+0000","last_clean":"2023-11-24T06:25:50.565479+0000","last_became_active":"2023-11-24T06:25:49.520985+0000","last_became_peered":"2023-11-24T06:25:49.520985+0000","last_unstale":"2023-11-24T06:25:50.565479+0000","last_undegraded":"2023-11-24T06:25:50.565479+0000","last_fullsized":"2023-11-24T06:25:50.565479+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:48.462279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528854+0000","last_change":"2023-11-24T06:25:50.548573+0000","last_active":"2023-11-24T06:25:51.528854+0000","last_peered":"2023-11-24T06:25:51.528854+0000","last_clean":"2023-11-24T06:25:51.528854+0000","last_became_active":"2023-11-24T06:25:50.548129+0000","last_became_peered":"2023-11-24T06:25:50.548129+0000","last_unstale":"2023-11-24T06:25:51.528854+0000","last_undegraded":"2023-11-24T06:25:51.528854+0000","last_fullsized":"2023-11-24T06:25:51.528854+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:39.732958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.572710+0000","last_change":"2023-11-24T06:25:49.531422+0000","last_active":"2023-11-24T06:25:50.572710+0000","last_peered":"2023-11-24T06:25:50.572710+0000","last_clean":"2023-11-24T06:25:50.572710+0000","last_became_active":"2023-11-24T06:25:49.531079+0000","last_became_peered":"2023-11-24T06:25:49.531079+0000","last_unstale":"2023-11-24T06:25:50.572710+0000","last_undegraded":"2023-11-24T06:25:50.572710+0000","last_fullsized":"2023-11-24T06:25:50.572710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:26.431372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528027+0000","last_change":"2023-11-24T06:25:50.536263+0000","last_active":"2023-11-24T06:25:51.528027+0000","last_peered":"2023-11-24T06:25:51.528027+0000","last_clean":"2023-11-24T06:25:51.528027+0000","last_became_active":"2023-11-24T06:25:50.535590+0000","last_became_peered":"2023-11-24T06:25:50.535590+0000","last_unstale":"2023-11-24T06:25:51.528027+0000","last_undegraded":"2023-11-24T06:25:51.528027+0000","last_fullsized":"2023-11-24T06:25:51.528027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:51.241710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527286+0000","last_change":"2023-11-24T06:25:49.531424+0000","last_active":"2023-11-24T06:25:50.527286+0000","last_peered":"2023-11-24T06:25:50.527286+0000","last_clean":"2023-11-24T06:25:50.527286+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:50.527286+0000","last_undegraded":"2023-11-24T06:25:50.527286+0000","last_fullsized":"2023-11-24T06:25:50.527286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:50.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385113+0000","last_change":"2023-11-24T06:25:50.554601+0000","last_active":"2023-11-24T06:25:52.385113+0000","last_peered":"2023-11-24T06:25:52.385113+0000","last_clean":"2023-11-24T06:25:52.385113+0000","last_became_active":"2023-11-24T06:25:50.554342+0000","last_became_peered":"2023-11-24T06:25:50.554342+0000","last_unstale":"2023-11-24T06:25:52.385113+0000","last_undegraded":"2023-11-24T06:25:52.385113+0000","last_fullsized":"2023-11-24T06:25:52.385113+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:24.373950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.539570+0000","last_change":"2023-11-24T06:25:49.531559+0000","last_active":"2023-11-24T06:25:50.539570+0000","last_peered":"2023-11-24T06:25:50.539570+0000","last_clean":"2023-11-24T06:25:50.539570+0000","last_became_active":"2023-11-24T06:25:49.531127+0000","last_became_peered":"2023-11-24T06:25:49.531127+0000","last_unstale":"2023-11-24T06:25:50.539570+0000","last_undegraded":"2023-11-24T06:25:50.539570+0000","last_fullsized":"2023-11-24T06:25:50.539570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:38.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529320+0000","last_change":"2023-11-24T06:25:50.544559+0000","last_active":"2023-11-24T06:25:51.529320+0000","last_peered":"2023-11-24T06:25:51.529320+0000","last_clean":"2023-11-24T06:25:51.529320+0000","last_became_active":"2023-11-24T06:25:50.544170+0000","last_became_peered":"2023-11-24T06:25:50.544170+0000","last_unstale":"2023-11-24T06:25:51.529320+0000","last_undegraded":"2023-11-24T06:25:51.529320+0000","last_fullsized":"2023-11-24T06:25:51.529320+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:43.722297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.556084+0000","last_change":"2023-11-24T06:25:49.529384+0000","last_active":"2023-11-24T06:25:50.556084+0000","last_peered":"2023-11-24T06:25:50.556084+0000","last_clean":"2023-11-24T06:25:50.556084+0000","last_became_active":"2023-11-24T06:25:49.529122+0000","last_became_peered":"2023-11-24T06:25:49.529122+0000","last_unstale":"2023-11-24T06:25:50.556084+0000","last_undegraded":"2023-11-24T06:25:50.556084+0000","last_fullsized":"2023-11-24T06:25:50.556084+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:09:11.371497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903879+0000","last_change":"2023-11-24T06:25:50.549166+0000","last_active":"2023-11-24T06:25:51.903879+0000","last_peered":"2023-11-24T06:25:51.903879+0000","last_clean":"2023-11-24T06:25:51.903879+0000","last_became_active":"2023-11-24T06:25:50.548877+0000","last_became_peered":"2023-11-24T06:25:50.548877+0000","last_unstale":"2023-11-24T06:25:51.903879+0000","last_undegraded":"2023-11-24T06:25:51.903879+0000","last_fullsized":"2023-11-24T06:25:51.903879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:43.375134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563151+0000","last_change":"2023-11-24T06:25:49.533140+0000","last_active":"2023-11-24T06:25:50.563151+0000","last_peered":"2023-11-24T06:25:50.563151+0000","last_clean":"2023-11-24T06:25:50.563151+0000","last_became_active":"2023-11-24T06:25:49.532965+0000","last_became_peered":"2023-11-24T06:25:49.532965+0000","last_unstale":"2023-11-24T06:25:50.563151+0000","last_undegraded":"2023-11-24T06:25:50.563151+0000","last_fullsized":"2023-11-24T06:25:50.563151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:04.680380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528311+0000","last_change":"2023-11-24T06:25:50.557006+0000","last_active":"2023-11-24T06:25:51.528311+0000","last_peered":"2023-11-24T06:25:51.528311+0000","last_clean":"2023-11-24T06:25:51.528311+0000","last_became_active":"2023-11-24T06:25:50.556741+0000","last_became_peered":"2023-11-24T06:25:50.556741+0000","last_unstale":"2023-11-24T06:25:51.528311+0000","last_undegraded":"2023-11-24T06:25:51.528311+0000","last_fullsized":"2023-11-24T06:25:51.528311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:31.443439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.558413+0000","last_change":"2023-11-24T06:25:49.522374+0000","last_active":"2023-11-24T06:25:50.558413+0000","last_peered":"2023-11-24T06:25:50.558413+0000","last_clean":"2023-11-24T06:25:50.558413+0000","last_became_active":"2023-11-24T06:25:49.522053+0000","last_became_peered":"2023-11-24T06:25:49.522053+0000","last_unstale":"2023-11-24T06:25:50.558413+0000","last_undegraded":"2023-11-24T06:25:50.558413+0000","last_fullsized":"2023-11-24T06:25:50.558413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:28.289197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528555+0000","last_change":"2023-11-24T06:25:50.554685+0000","last_active":"2023-11-24T06:25:51.528555+0000","last_peered":"2023-11-24T06:25:51.528555+0000","last_clean":"2023-11-24T06:25:51.528555+0000","last_became_active":"2023-11-24T06:25:50.554379+0000","last_became_peered":"2023-11-24T06:25:50.554379+0000","last_unstale":"2023-11-24T06:25:51.528555+0000","last_undegraded":"2023-11-24T06:25:51.528555+0000","last_fullsized":"2023-11-24T06:25:51.528555+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:22.188611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.558291+0000","last_change":"2023-11-24T06:25:49.528898+0000","last_active":"2023-11-24T06:25:50.558291+0000","last_peered":"2023-11-24T06:25:50.558291+0000","last_clean":"2023-11-24T06:25:50.558291+0000","last_became_active":"2023-11-24T06:25:49.528642+0000","last_became_peered":"2023-11-24T06:25:49.528642+0000","last_unstale":"2023-11-24T06:25:50.558291+0000","last_undegraded":"2023-11-24T06:25:50.558291+0000","last_fullsized":"2023-11-24T06:25:50.558291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:42.438757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.529524+0000","last_change":"2023-11-24T06:25:50.541711+0000","last_active":"2023-11-24T06:25:51.529524+0000","last_peered":"2023-11-24T06:25:51.529524+0000","last_clean":"2023-11-24T06:25:51.529524+0000","last_became_active":"2023-11-24T06:25:50.541407+0000","last_became_peered":"2023-11-24T06:25:50.541407+0000","last_unstale":"2023-11-24T06:25:51.529524+0000","last_undegraded":"2023-11-24T06:25:51.529524+0000","last_fullsized":"2023-11-24T06:25:51.529524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:10.604834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.540988+0000","last_change":"2023-11-24T06:25:49.531568+0000","last_active":"2023-11-24T06:25:50.540988+0000","last_peered":"2023-11-24T06:25:50.540988+0000","last_clean":"2023-11-24T06:25:50.540988+0000","last_became_active":"2023-11-24T06:25:49.531275+0000","last_became_peered":"2023-11-24T06:25:49.531275+0000","last_unstale":"2023-11-24T06:25:50.540988+0000","last_undegraded":"2023-11-24T06:25:50.540988+0000","last_fullsized":"2023-11-24T06:25:50.540988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:15:59.158530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385319+0000","last_change":"2023-11-24T06:25:50.553768+0000","last_active":"2023-11-24T06:25:52.385319+0000","last_peered":"2023-11-24T06:25:52.385319+0000","last_clean":"2023-11-24T06:25:52.385319+0000","last_became_active":"2023-11-24T06:25:50.553479+0000","last_became_peered":"2023-11-24T06:25:50.553479+0000","last_unstale":"2023-11-24T06:25:52.385319+0000","last_undegraded":"2023-11-24T06:25:52.385319+0000","last_fullsized":"2023-11-24T06:25:52.385319+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:59.380344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563066+0000","last_change":"2023-11-24T06:25:49.531979+0000","last_active":"2023-11-24T06:25:50.563066+0000","last_peered":"2023-11-24T06:25:50.563066+0000","last_clean":"2023-11-24T06:25:50.563066+0000","last_became_active":"2023-11-24T06:25:49.531699+0000","last_became_peered":"2023-11-24T06:25:49.531699+0000","last_unstale":"2023-11-24T06:25:50.563066+0000","last_undegraded":"2023-11-24T06:25:50.563066+0000","last_fullsized":"2023-11-24T06:25:50.563066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:51.611145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385893+0000","last_change":"2023-11-24T06:25:50.547085+0000","last_active":"2023-11-24T06:25:52.385893+0000","last_peered":"2023-11-24T06:25:52.385893+0000","last_clean":"2023-11-24T06:25:52.385893+0000","last_became_active":"2023-11-24T06:25:50.546821+0000","last_became_peered":"2023-11-24T06:25:50.546821+0000","last_unstale":"2023-11-24T06:25:52.385893+0000","last_undegraded":"2023-11-24T06:25:52.385893+0000","last_fullsized":"2023-11-24T06:25:52.385893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:10.604834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.533396+0000","last_change":"2023-11-24T06:25:49.528142+0000","last_active":"2023-11-24T06:25:50.533396+0000","last_peered":"2023-11-24T06:25:50.533396+0000","last_clean":"2023-11-24T06:25:50.533396+0000","last_became_active":"2023-11-24T06:25:49.527836+0000","last_became_peered":"2023-11-24T06:25:49.527836+0000","last_unstale":"2023-11-24T06:25:50.533396+0000","last_undegraded":"2023-11-24T06:25:50.533396+0000","last_fullsized":"2023-11-24T06:25:50.533396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:41.705584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.903708+0000","last_change":"2023-11-24T06:25:50.543720+0000","last_active":"2023-11-24T06:25:51.903708+0000","last_peered":"2023-11-24T06:25:51.903708+0000","last_clean":"2023-11-24T06:25:51.903708+0000","last_became_active":"2023-11-24T06:25:50.543445+0000","last_became_peered":"2023-11-24T06:25:50.543445+0000","last_unstale":"2023-11-24T06:25:51.903708+0000","last_undegraded":"2023-11-24T06:25:51.903708+0000","last_fullsized":"2023-11-24T06:25:51.903708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:43.331737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.530249+0000","last_change":"2023-11-24T06:25:49.524448+0000","last_active":"2023-11-24T06:25:50.530249+0000","last_peered":"2023-11-24T06:25:50.530249+0000","last_clean":"2023-11-24T06:25:50.530249+0000","last_became_active":"2023-11-24T06:25:49.524159+0000","last_became_peered":"2023-11-24T06:25:49.524159+0000","last_unstale":"2023-11-24T06:25:50.530249+0000","last_undegraded":"2023-11-24T06:25:50.530249+0000","last_fullsized":"2023-11-24T06:25:50.530249+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:09.597215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527814+0000","last_change":"2023-11-24T06:25:50.550108+0000","last_active":"2023-11-24T06:25:51.527814+0000","last_peered":"2023-11-24T06:25:51.527814+0000","last_clean":"2023-11-24T06:25:51.527814+0000","last_became_active":"2023-11-24T06:25:50.549838+0000","last_became_peered":"2023-11-24T06:25:50.549838+0000","last_unstale":"2023-11-24T06:25:51.527814+0000","last_undegraded":"2023-11-24T06:25:51.527814+0000","last_fullsized":"2023-11-24T06:25:51.527814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:14.915316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.526972+0000","last_change":"2023-11-24T06:25:49.531285+0000","last_active":"2023-11-24T06:25:50.526972+0000","last_peered":"2023-11-24T06:25:50.526972+0000","last_clean":"2023-11-24T06:25:50.526972+0000","last_became_active":"2023-11-24T06:25:49.530974+0000","last_became_peered":"2023-11-24T06:25:49.530974+0000","last_unstale":"2023-11-24T06:25:50.526972+0000","last_undegraded":"2023-11-24T06:25:50.526972+0000","last_fullsized":"2023-11-24T06:25:50.526972+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:07.434705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528783+0000","last_change":"2023-11-24T06:25:50.548580+0000","last_active":"2023-11-24T06:25:51.528783+0000","last_peered":"2023-11-24T06:25:51.528783+0000","last_clean":"2023-11-24T06:25:51.528783+0000","last_became_active":"2023-11-24T06:25:50.548174+0000","last_became_peered":"2023-11-24T06:25:50.548174+0000","last_unstale":"2023-11-24T06:25:51.528783+0000","last_undegraded":"2023-11-24T06:25:51.528783+0000","last_fullsized":"2023-11-24T06:25:51.528783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:22.232490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.540695+0000","last_change":"2023-11-24T06:25:49.534058+0000","last_active":"2023-11-24T06:25:50.540695+0000","last_peered":"2023-11-24T06:25:50.540695+0000","last_clean":"2023-11-24T06:25:50.540695+0000","last_became_active":"2023-11-24T06:25:49.533728+0000","last_became_peered":"2023-11-24T06:25:49.533728+0000","last_unstale":"2023-11-24T06:25:50.540695+0000","last_undegraded":"2023-11-24T06:25:50.540695+0000","last_fullsized":"2023-11-24T06:25:50.540695+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:41.828333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.527993+0000","last_change":"2023-11-24T06:25:50.535053+0000","last_active":"2023-11-24T06:25:51.527993+0000","last_peered":"2023-11-24T06:25:51.527993+0000","last_clean":"2023-11-24T06:25:51.527993+0000","last_became_active":"2023-11-24T06:25:50.534759+0000","last_became_peered":"2023-11-24T06:25:50.534759+0000","last_unstale":"2023-11-24T06:25:51.527993+0000","last_undegraded":"2023-11-24T06:25:51.527993+0000","last_fullsized":"2023-11-24T06:25:51.527993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:06.032977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528873+0000","last_change":"2023-11-24T06:25:49.526982+0000","last_active":"2023-11-24T06:25:50.528873+0000","last_peered":"2023-11-24T06:25:50.528873+0000","last_clean":"2023-11-24T06:25:50.528873+0000","last_became_active":"2023-11-24T06:25:49.526688+0000","last_became_peered":"2023-11-24T06:25:49.526688+0000","last_unstale":"2023-11-24T06:25:50.528873+0000","last_undegraded":"2023-11-24T06:25:50.528873+0000","last_fullsized":"2023-11-24T06:25:50.528873+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:43.762747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528792+0000","last_change":"2023-11-24T06:25:50.547067+0000","last_active":"2023-11-24T06:25:51.528792+0000","last_peered":"2023-11-24T06:25:51.528792+0000","last_clean":"2023-11-24T06:25:51.528792+0000","last_became_active":"2023-11-24T06:25:50.546738+0000","last_became_peered":"2023-11-24T06:25:50.546738+0000","last_unstale":"2023-11-24T06:25:51.528792+0000","last_undegraded":"2023-11-24T06:25:51.528792+0000","last_fullsized":"2023-11-24T06:25:51.528792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:39.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.536560+0000","last_change":"2023-11-24T06:25:49.529663+0000","last_active":"2023-11-24T06:25:50.536560+0000","last_peered":"2023-11-24T06:25:50.536560+0000","last_clean":"2023-11-24T06:25:50.536560+0000","last_became_active":"2023-11-24T06:25:49.529401+0000","last_became_peered":"2023-11-24T06:25:49.529401+0000","last_unstale":"2023-11-24T06:25:50.536560+0000","last_undegraded":"2023-11-24T06:25:50.536560+0000","last_fullsized":"2023-11-24T06:25:50.536560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:15.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385715+0000","last_change":"2023-11-24T06:25:50.557619+0000","last_active":"2023-11-24T06:25:52.385715+0000","last_peered":"2023-11-24T06:25:52.385715+0000","last_clean":"2023-11-24T06:25:52.385715+0000","last_became_active":"2023-11-24T06:25:50.557331+0000","last_became_peered":"2023-11-24T06:25:50.557331+0000","last_unstale":"2023-11-24T06:25:52.385715+0000","last_undegraded":"2023-11-24T06:25:52.385715+0000","last_fullsized":"2023-11-24T06:25:52.385715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:25.669514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527569+0000","last_change":"2023-11-24T06:25:49.525526+0000","last_active":"2023-11-24T06:25:50.527569+0000","last_peered":"2023-11-24T06:25:50.527569+0000","last_clean":"2023-11-24T06:25:50.527569+0000","last_became_active":"2023-11-24T06:25:49.525235+0000","last_became_peered":"2023-11-24T06:25:49.525235+0000","last_unstale":"2023-11-24T06:25:50.527569+0000","last_undegraded":"2023-11-24T06:25:50.527569+0000","last_fullsized":"2023-11-24T06:25:50.527569+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:58.851781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.385368+0000","last_change":"2023-11-24T06:25:50.557396+0000","last_active":"2023-11-24T06:25:52.385368+0000","last_peered":"2023-11-24T06:25:52.385368+0000","last_clean":"2023-11-24T06:25:52.385368+0000","last_became_active":"2023-11-24T06:25:50.557136+0000","last_became_peered":"2023-11-24T06:25:50.557136+0000","last_unstale":"2023-11-24T06:25:52.385368+0000","last_undegraded":"2023-11-24T06:25:52.385368+0000","last_fullsized":"2023-11-24T06:25:52.385368+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:01.760795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528433+0000","last_change":"2023-11-24T06:25:49.520031+0000","last_active":"2023-11-24T06:25:50.528433+0000","last_peered":"2023-11-24T06:25:50.528433+0000","last_clean":"2023-11-24T06:25:50.528433+0000","last_became_active":"2023-11-24T06:25:49.519789+0000","last_became_peered":"2023-11-24T06:25:49.519789+0000","last_unstale":"2023-11-24T06:25:50.528433+0000","last_undegraded":"2023-11-24T06:25:50.528433+0000","last_fullsized":"2023-11-24T06:25:50.528433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:15.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.770644+0000","last_change":"2023-11-24T06:25:50.553590+0000","last_active":"2023-11-24T06:25:51.770644+0000","last_peered":"2023-11-24T06:25:51.770644+0000","last_clean":"2023-11-24T06:25:51.770644+0000","last_became_active":"2023-11-24T06:25:50.553224+0000","last_became_peered":"2023-11-24T06:25:50.553224+0000","last_unstale":"2023-11-24T06:25:51.770644+0000","last_undegraded":"2023-11-24T06:25:51.770644+0000","last_fullsized":"2023-11-24T06:25:51.770644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:31.443439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.536706+0000","last_change":"2023-11-24T06:25:49.525157+0000","last_active":"2023-11-24T06:25:50.536706+0000","last_peered":"2023-11-24T06:25:50.536706+0000","last_clean":"2023-11-24T06:25:50.536706+0000","last_became_active":"2023-11-24T06:25:49.524896+0000","last_became_peered":"2023-11-24T06:25:49.524896+0000","last_unstale":"2023-11-24T06:25:50.536706+0000","last_undegraded":"2023-11-24T06:25:50.536706+0000","last_fullsized":"2023-11-24T06:25:50.536706+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:12.705475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:52.384926+0000","last_change":"2023-11-24T06:25:50.547601+0000","last_active":"2023-11-24T06:25:52.384926+0000","last_peered":"2023-11-24T06:25:52.384926+0000","last_clean":"2023-11-24T06:25:52.384926+0000","last_became_active":"2023-11-24T06:25:50.547298+0000","last_became_peered":"2023-11-24T06:25:50.547298+0000","last_unstale":"2023-11-24T06:25:52.384926+0000","last_undegraded":"2023-11-24T06:25:52.384926+0000","last_fullsized":"2023-11-24T06:25:52.384926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:45:51.885765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.528635+0000","last_change":"2023-11-24T06:25:49.528916+0000","last_active":"2023-11-24T06:25:50.528635+0000","last_peered":"2023-11-24T06:25:50.528635+0000","last_clean":"2023-11-24T06:25:50.528635+0000","last_became_active":"2023-11-24T06:25:49.528145+0000","last_became_peered":"2023-11-24T06:25:49.528145+0000","last_unstale":"2023-11-24T06:25:50.528635+0000","last_undegraded":"2023-11-24T06:25:50.528635+0000","last_fullsized":"2023-11-24T06:25:50.528635+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:36.494777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:25:51.528149+0000","last_change":"2023-11-24T06:25:50.532640+0000","last_active":"2023-11-24T06:25:51.528149+0000","last_peered":"2023-11-24T06:25:51.528149+0000","last_clean":"2023-11-24T06:25:51.528149+0000","last_became_active":"2023-11-24T06:25:50.532381+0000","last_became_peered":"2023-11-24T06:25:50.532381+0000","last_unstale":"2023-11-24T06:25:51.528149+0000","last_undegraded":"2023-11-24T06:25:51.528149+0000","last_fullsized":"2023-11-24T06:25:51.528149+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:49.505313+0000","last_clean_scrub_stamp":"2023-11-24T06:25:49.505313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:57.639964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.563178+0000","last_change":"2023-11-24T06:25:49.533010+0000","last_active":"2023-11-24T06:25:50.563178+0000","last_peered":"2023-11-24T06:25:50.563178+0000","last_clean":"2023-11-24T06:25:50.563178+0000","last_became_active":"2023-11-24T06:25:49.532766+0000","last_became_peered":"2023-11-24T06:25:49.532766+0000","last_unstale":"2023-11-24T06:25:50.563178+0000","last_undegraded":"2023-11-24T06:25:50.563178+0000","last_fullsized":"2023-11-24T06:25:50.563178+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:25:48.497694+0000","last_clean_scrub_stamp":"2023-11-24T06:25:48.497694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:50.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:25:50.527082+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:25:50.527082+0000","last_peered":"2023-11-24T06:25:50.527082+0000","last_clean":"2023-11-24T06:25:50.527082+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:25:50.527082+0000","last_undegraded":"2023-11-24T06:25:50.527082+0000","last_fullsized":"2023-11-24T06:25:50.527082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":804905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.501},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":7.9340000000000002,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":8.5890000000000004,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.39100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.308},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":6.9189999999999996,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.499},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.377}]},{"osd":3,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.436},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:25:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.55500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":804849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":11.44,"5min":11.44,"15min":11.44},"last":0.253},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.0449999999999999,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.157,"5min":11.157,"15min":11.157},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.4630000000000001,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.373}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":11.378,"5min":11.378,"15min":11.378},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":11.343,"5min":11.343,"15min":11.343},"last":0.622}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":11.407999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":216081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:25:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.77300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":215085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":10.052,"5min":10.052,"15min":10.052},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.314},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.377},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":10.077999999999999,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":9.9459999999999997,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.7919999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.621},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.9939999999999998,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":216059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":0.316},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.625}]},{"osd":3,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.373},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.36099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.438}]},{"osd":7,"last update":"Fri Nov 24 06:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:25:59.563 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:25:59.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:25:59.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.723+0000 7f309ba08700 1 Processor -- start 2023-11-24T06:25:59.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.723+0000 7f309ba08700 1 -- start start 2023-11-24T06:25:59.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f309ba08700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f30940ad870 0x7f30940ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f309ba08700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309406e400 con 0x7f30940aedc0 2023-11-24T06:25:59.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f309ba08700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309406e630 con 0x7f309415e290 2023-11-24T06:25:59.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f309ba08700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309406fde0 con 0x7f30940ad870 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f3099fa5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f30940ad870 0x7f30940ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f30997a4700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f30940aedc0 0x7f30940af1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56668/0 (socket says 172.21.15.114:56668) 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f30997a4700 1 -- 172.21.15.114:0/301376379 learned_addr learned my addr 172.21.15.114:0/301376379 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f3099fa5700 1 -- 172.21.15.114:0/301376379 >> v1:172.21.15.172:6789/0 conn(0x7f309415e290 legacy=0x7f309406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f3099fa5700 1 -- 172.21.15.114:0/301376379 >> v1:172.21.15.114:6789/0 conn(0x7f30940aedc0 legacy=0x7f30940af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f3099fa5700 1 -- 172.21.15.114:0/301376379 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309406e910 con 0x7f30940ad870 2023-11-24T06:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f308bfff700 1 -- 172.21.15.114:0/301376379 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175494650 0 0) 0x7f309406e400 con 0x7f30940aedc0 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f3099fa5700 1 --2- 172.21.15.114:0/301376379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30940ae790 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f308c009d90 tx=0x7f308c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c53f25e65671fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f308bfff700 1 -- 172.21.15.114:0/301376379 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00e070 con 0x7f30940ad870 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f308bfff700 1 -- 172.21.15.114:0/301376379 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c00bbd0 con 0x7f30940ad870 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.724+0000 7f308bfff700 1 -- 172.21.15.114:0/301376379 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c012430 con 0x7f30940ad870 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 -- 172.21.15.114:0/301376379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 msgr2=0x7f30940ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 --2- 172.21.15.114:0/301376379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30940ae790 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f308c009d90 tx=0x7f308c00b330 comp rx=0 tx=0).stop 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 -- 172.21.15.114:0/301376379 shutdown_connections 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 --2- 172.21.15.114:0/301376379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30940ae790 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 -- 172.21.15.114:0/301376379 >> 172.21.15.114:0/301376379 conn(0x7f30940b2050 msgr2=0x7f30940b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:25:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 -- 172.21.15.114:0/301376379 shutdown_connections 2023-11-24T06:25:59.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.725+0000 7f309ba08700 1 -- 172.21.15.114:0/301376379 wait complete. 2023-11-24T06:25:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.726+0000 7f309ba08700 1 Processor -- start 2023-11-24T06:25:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.726+0000 7f309ba08700 1 -- start start 2023-11-24T06:25:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30940aedc0 0x7f3094157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 0x7f3094143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309415eb10 con 0x7f30940ad870 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309415edf0 con 0x7f30940aedc0 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f309ba08700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309415f0d0 con 0x7f309415e290 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40970/0 (socket says 172.21.15.114:40970) 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 -- 172.21.15.114:0/2433954238 learned_addr learned my addr 172.21.15.114:0/2433954238 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f3099fa5700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 0x7f3094143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f3098fa3700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30940aedc0 0x7f3094157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 -- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 msgr2=0x7f3094143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 0x7f3094143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 -- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30940aedc0 msgr2=0x7f3094157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:25:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30940aedc0 0x7f3094157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f30997a4700 1 -- 172.21.15.114:0/2433954238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c009a70 con 0x7f30940ad870 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.727+0000 7f3099fa5700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 0x7f3094143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f30997a4700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f309000c520 tx=0x7f309000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090004c80 con 0x7f30940ad870 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090004e00 con 0x7f30940ad870 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309415f3b0 con 0x7f30940ad870 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090007310 con 0x7f30940ad870 2023-11-24T06:25:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.728+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3094160690 con 0x7f30940ad870 2023-11-24T06:25:59.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.729+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3094068d90 con 0x7f30940ad870 2023-11-24T06:25:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.729+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3090007470 con 0x7f30940ad870 2023-11-24T06:25:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.729+0000 7f3089ffb700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 0x7f307806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:25:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.730+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3090089620 con 0x7f30940ad870 2023-11-24T06:25:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.730+0000 7f3098fa3700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 0x7f307806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:25:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.730+0000 7f3098fa3700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 0x7f307806d540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3084000f60 tx=0x7f3084009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:25:59.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:25:59.734+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f30900544a0 con 0x7f30940ad870 2023-11-24T06:26:00.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.004+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f309415ef30 con 0x7f30940ad870 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.007+0000 7f3089ffb700 1 -- 172.21.15.114:0/2433954238 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f309415ef30 con 0x7f30940ad870 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g== 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stdout: caps mds = "allow" 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stdout: caps mon = "allow r" 2023-11-24T06:26:00.010 INFO:teuthology.orchestra.run.smithi114.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 msgr2=0x7f307806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 0x7f307806d540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3084000f60 tx=0x7f3084009040 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 msgr2=0x7f30941f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f309000c520 tx=0x7f309000c8e0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 shutdown_connections 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f307806b080 0x7f307806d540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f309415e290 0x7f3094143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30940ad870 0x7f30941f7a50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 --2- 172.21.15.114:0/2433954238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30940aedc0 0x7f3094157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 >> 172.21.15.114:0/2433954238 conn(0x7f30940b2050 msgr2=0x7f3094159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 shutdown_connections 2023-11-24T06:26:00.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.009+0000 7f309ba08700 1 -- 172.21.15.114:0/2433954238 wait complete. 2023-11-24T06:26:00.014 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.0 2023-11-24T06:26:00.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.141+0000 7f91c7c3a700 1 Processor -- start 2023-11-24T06:26:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.141+0000 7f91c7c3a700 1 -- start start 2023-11-24T06:26:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c7c3a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91c00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c7c3a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c015c540 con 0x7f91c00b04b0 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c7c3a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c00b2ee0 con 0x7f91c00b2870 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c7c3a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00b2cf0 con 0x7f91c015c160 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c61d7700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f91c00b04b0 0x7f91c0165c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56686/0 (socket says 172.21.15.114:56686) 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c61d7700 1 -- 172.21.15.114:0/508341242 learned_addr learned my addr 172.21.15.114:0/508341242 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712511596 0 0) 0x7f91c015c540 con 0x7f91c00b04b0 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c59d6700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91c00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.142+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ac0035b0 con 0x7f91c00b04b0 2023-11-24T06:26:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820201455 0 0) 0x7f91ac0035b0 con 0x7f91c00b04b0 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 msgr2=0x7f91c00622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91c00622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 >> v1:172.21.15.172:6789/0 conn(0x7f91c00b2870 legacy=0x7f91c00afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00add40 con 0x7f91c00b04b0 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f91b80019f0 con 0x7f91c00b04b0 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 >> v1:172.21.15.114:6789/0 conn(0x7f91c00b04b0 legacy=0x7f91c0165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac003a30 0x7f91ac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ac004850 0x7f91ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00bac0 con 0x7f91ac003a30 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00bc00 con 0x7f91ac004850 2023-11-24T06:26:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00bdc0 con 0x7f91c015c160 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac003a30 0x7f91ac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c51d5700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ac004850 0x7f91ac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c61d7700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 -- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 msgr2=0x7f91ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91c015c160 0x7f91ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 -- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ac004850 msgr2=0x7f91ac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91ac004850 0x7f91ac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 -- 172.21.15.114:0/508341242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0035b0 con 0x7f91ac003a30 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c59d6700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac003a30 0x7f91ac003e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009b30 tx=0x7f91ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44571fce5676e83e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000e6c0 con 0x7f91ac003a30 2023-11-24T06:26:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b000e820 con 0x7f91ac003a30 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.143+0000 7f91c49d4700 1 -- 172.21.15.114:0/508341242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000eb90 con 0x7f91ac003a30 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 -- 172.21.15.114:0/508341242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac003a30 msgr2=0x7f91ac003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/508341242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac003a30 0x7f91ac003e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009b30 tx=0x7f91ac005880 comp rx=0 tx=0).stop 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c61d7700 1 -- 172.21.15.114:0/508341242 reap_dead start 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 -- 172.21.15.114:0/508341242 shutdown_connections 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 -- 172.21.15.114:0/508341242 >> 172.21.15.114:0/508341242 conn(0x7f91c00b57b0 msgr2=0x7f91c00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 -- 172.21.15.114:0/508341242 shutdown_connections 2023-11-24T06:26:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.144+0000 7f91c7c3a700 1 -- 172.21.15.114:0/508341242 wait complete. 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.145+0000 7f91c7c3a700 1 Processor -- start 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.145+0000 7f91c7c3a700 1 -- start start 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ac003a30 0x7f91c015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91c01513a0 0x7f91c014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c015f560 con 0x7f91c01f5a80 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c0151d80 con 0x7f91c01513a0 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c7c3a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00cbd80 con 0x7f91ac003a30 2023-11-24T06:26:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40988/0 (socket says 172.21.15.114:40988) 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 -- 172.21.15.114:0/2906871495 learned_addr learned my addr 172.21.15.114:0/2906871495 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c61d7700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91c01513a0 0x7f91c014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c59d6700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ac003a30 0x7f91c015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 -- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ac003a30 msgr2=0x7f91c015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ac003a30 0x7f91c015f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 -- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91c01513a0 msgr2=0x7f91c014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91c01513a0 0x7f91c014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.146+0000 7f91c51d5700 1 -- 172.21.15.114:0/2906871495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00cc060 con 0x7f91c01f5a80 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91c51d5700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e550 tx=0x7f91bc00e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0052e0 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc005440 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c00cc340 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0038a0 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.147+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c013e7c0 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.148+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c013e980 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.148+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f91bc003a00 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.149+0000 7f91b67fc700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 0x7f91a406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.149+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91bc088cf0 con 0x7f91c01f5a80 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.149+0000 7f91c59d6700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 0x7f91a406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.149+0000 7f91c59d6700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 0x7f91a406d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91b00090b0 tx=0x7f91b001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.153+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91bc053b80 con 0x7f91c01f5a80 2023-11-24T06:26:00.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.386+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91c00b2ee0 con 0x7f91c01f5a80 2023-11-24T06:26:00.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.386+0000 7f91b67fc700 1 -- 172.21.15.114:0/2906871495 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f91c00b2ee0 con 0x7f91c01f5a80 2023-11-24T06:26:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:00.390 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 msgr2=0x7f91a406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 0x7f91a406d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91b00090b0 tx=0x7f91b001f040 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 msgr2=0x7f91c013e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00e550 tx=0x7f91bc00e860 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 shutdown_connections 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f91a406b080 0x7f91a406d540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91c01513a0 0x7f91c014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91ac003a30 0x7f91c015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 --2- 172.21.15.114:0/2906871495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91c01f5a80 0x7f91c013e280 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 >> 172.21.15.114:0/2906871495 conn(0x7f91c00b57b0 msgr2=0x7f91c00ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 shutdown_connections 2023-11-24T06:26:00.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.388+0000 7f91c7c3a700 1 -- 172.21.15.114:0/2906871495 wait complete. 2023-11-24T06:26:00.393 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 27 2023-11-24T06:26:00.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:25:49.913212+0000', 'modified': '2023-11-24T06:25:52.939341+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5646}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5646': {'gid': 5646, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6837/4123842361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6835', 'nonce': 4123842361}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 4123842361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:26:00.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:26:00.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:26:00.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.932+0000 7f9f17716700 1 Processor -- start 2023-11-24T06:26:00.935 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.932+0000 7f9f17716700 1 -- start start 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f17716700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f1015e080 0x7f9f100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f17716700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f1006c710 0x7f9f100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f17716700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100ae430 con 0x7f9f100af4d0 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f17716700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f100af0b0 con 0x7f9f1006c710 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f17716700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f100b05a0 con 0x7f9f1015e080 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f1006c710 0x7f9f100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9f1006c710 0x7f9f100afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40996/0 (socket says 172.21.15.114:40996) 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 -- 172.21.15.114:0/1902914844 learned_addr learned my addr 172.21.15.114:0/1902914844 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f154b2700 1 --2- 172.21.15.114:0/1902914844 >> v2:172.21.15.172:3300/0 conn(0x7f9f1015e080 0x7f9f100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 -- 172.21.15.114:0/1902914844 >> v2:172.21.15.172:3300/0 conn(0x7f9f1015e080 msgr2=0x7f9f100b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.938 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/1902914844 >> v2:172.21.15.172:3300/0 conn(0x7f9f1015e080 0x7f9f100b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 -- 172.21.15.114:0/1902914844 >> v1:172.21.15.172:6789/0 conn(0x7f9f100af4d0 legacy=0x7f9f1006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 -- 172.21.15.114:0/1902914844 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100ae710 con 0x7f9f1006c710 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/1902914844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1006c710 0x7f9f100afb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f10072a90 tx=0x7f9f0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9209829894962aec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f07fff700 1 -- 172.21.15.114:0/1902914844 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800e070 con 0x7f9f1006c710 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f07fff700 1 -- 172.21.15.114:0/1902914844 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f08003a20 con 0x7f9f1006c710 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.933+0000 7f9f07fff700 1 -- 172.21.15.114:0/1902914844 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08003dd0 con 0x7f9f1006c710 2023-11-24T06:26:00.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 -- 172.21.15.114:0/1902914844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1006c710 msgr2=0x7f9f100afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 --2- 172.21.15.114:0/1902914844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1006c710 0x7f9f100afb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f10072a90 tx=0x7f9f0800b330 comp rx=0 tx=0).stop 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 -- 172.21.15.114:0/1902914844 shutdown_connections 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 --2- 172.21.15.114:0/1902914844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1006c710 0x7f9f100afb90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 --2- 172.21.15.114:0/1902914844 >> v2:172.21.15.172:3300/0 conn(0x7f9f1015e080 0x7f9f100b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 -- 172.21.15.114:0/1902914844 >> 172.21.15.114:0/1902914844 conn(0x7f9f10071b90 msgr2=0x7f9f10071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 -- 172.21.15.114:0/1902914844 shutdown_connections 2023-11-24T06:26:00.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.934+0000 7f9f17716700 1 -- 172.21.15.114:0/1902914844 wait complete. 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.935+0000 7f9f17716700 1 Processor -- start 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.935+0000 7f9f17716700 1 -- start start 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f1006c710 0x7f9f1014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f100af4d0 0x7f9f101f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100ce6d0 con 0x7f9f1015e080 2023-11-24T06:26:00.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100ce9b0 con 0x7f9f1006c710 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f17716700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100cec90 con 0x7f9f100af4d0 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41002/0 (socket says 172.21.15.114:41002) 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 -- 172.21.15.114:0/497908869 learned_addr learned my addr 172.21.15.114:0/497908869 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f100af4d0 0x7f9f101f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f154b2700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f1006c710 0x7f9f1014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 -- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f100af4d0 msgr2=0x7f9f101f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f100af4d0 0x7f9f101f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 -- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f1006c710 msgr2=0x7f9f1014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f1006c710 0x7f9f1014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 -- 172.21.15.114:0/497908869 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f08009b30 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.936+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f08004000 tx=0x7f9f08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.937+0000 7f9f05ffb700 1 -- 172.21.15.114:0/497908869 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004440 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.937+0000 7f9f05ffb700 1 -- 172.21.15.114:0/497908869 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f080045e0 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.937+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f100cef70 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.937+0000 7f9f05ffb700 1 -- 172.21.15.114:0/497908869 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080094d0 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.937+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f101f3b30 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.938+0000 7f9f05ffb700 1 -- 172.21.15.114:0/497908869 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9f08012430 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.938+0000 7f9f05ffb700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 0x7f9ef406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.939+0000 7f9f05ffb700 1 -- 172.21.15.114:0/497908869 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9f0808ae40 con 0x7f9f1015e080 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.939+0000 7f9f154b2700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 0x7f9ef406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.939+0000 7f9f154b2700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 0x7f9ef406d540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000b9b0 tx=0x7f9f00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.940+0000 7f9f2390fb80 1 Processor -- start 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 -- start start 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8d080 0x5640b0ed3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8a490 0x5640b0eb6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0e00770 con 0x5640b0ea2ed0 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0de7750 con 0x5640b0e8d080 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640b0de5070 con 0x5640b0e8a490 2023-11-24T06:26:00.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41008/0 (socket says 172.21.15.114:41008) 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.941+0000 7f9f14cb1700 1 -- 172.21.15.114:0/2473804755 learned_addr learned my addr 172.21.15.114:0/2473804755 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8a490 0x5640b0eb6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f154b2700 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8d080 0x5640b0ed3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 -- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8a490 msgr2=0x5640b0eb6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8a490 0x5640b0eb6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 -- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8d080 msgr2=0x5640b0ed3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8d080 0x5640b0ed3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 -- 172.21.15.114:0/2473804755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b0c5d5e0 con 0x5640b0ea2ed0 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c004db0 tx=0x7f9f0c0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fd9ffc662eda08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9efaffd700 1 -- 172.21.15.114:0/2473804755 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c006a70 con 0x5640b0ea2ed0 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9efaffd700 1 -- 172.21.15.114:0/2473804755 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c006bd0 con 0x5640b0ea2ed0 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9efaffd700 1 -- 172.21.15.114:0/2473804755 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c007030 con 0x5640b0ea2ed0 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f2390fb80 1 -- 172.21.15.114:0/2473804755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 msgr2=0x5640b0ed3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.942+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c004db0 tx=0x7f9f0c0050c0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 -- 172.21.15.114:0/2473804755 shutdown_connections 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8a490 0x5640b0eb6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8d080 0x5640b0ed3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/2473804755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0ea2ed0 0x5640b0ed3d80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 -- 172.21.15.114:0/2473804755 >> 172.21.15.114:0/2473804755 conn(0x5640b0dbb310 msgr2=0x5640b0ed1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 -- 172.21.15.114:0/2473804755 shutdown_connections 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 -- 172.21.15.114:0/2473804755 wait complete. 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 -- start start 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.943+0000 7f9f2390fb80 1 Processor -- start 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 0x5640b0ecf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0ea2ed0 0x5640b0eba300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0de0960 con 0x5640b0e8a490 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0d9a510 con 0x5640b0ea2ed0 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640b0d9a1b0 con 0x5640b0e8d080 2023-11-24T06:26:00.955 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f2390fb80 10 client.0 _ll_register_callbacks cb 0x5640b0cc5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f154b2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f154b2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41012/0 (socket says 172.21.15.114:41012) 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f154b2700 1 -- 172.21.15.114:0/892812685 learned_addr learned my addr 172.21.15.114:0/892812685 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 0x5640b0ecf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.944+0000 7f9f15cb3700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0ea2ed0 0x5640b0eba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 -- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 msgr2=0x5640b0ecf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 0x5640b0ecf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 -- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0ea2ed0 msgr2=0x5640b0eba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0ea2ed0 0x5640b0eba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 -- 172.21.15.114:0/892812685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c00f040 con 0x5640b0e8a490 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 0x5640b0ecf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f154b2700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000ff70 tx=0x7f9f000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f64e9eab6df40c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00004020 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9f00004180 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b0e7fa40 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f00004340 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 10 client.5826 fetch_fsmap learned FSMap version 27 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5640b0c09d90 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640b0e7fa40 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f000036a0 con 0x5640b0e8a490 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 10 client.5826 fetch_fsmap finished waiting for FSMap version 27 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 10 client.5826 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:26:00.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 20 client.5826 populate_metadata read hostname 'smithi114' 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ea25f0 0x5640b0ee4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 4 client.5826 mds_command: new command op to 5646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 --> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640b0e5b840 con 0x5640b0ea25f0 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.945+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ea25f0 0x5640b0ee4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9eeffff700 10 client.5826 ms_handle_connect on v2:172.21.15.114:6835/4123842361 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ea25f0 0x5640b0ee4790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83a6db42709ddd7 server_cookie=16bb282fd7aa86d8 in_seq=0 out_seq=0 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9eeffff700 1 -- 172.21.15.114:0/892812685 <== mds.0 v2:172.21.15.114:6835/4123842361 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640b0e5b840 con 0x5640b0ea25f0 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9eeffff700 10 client.5826 handle_command_reply: tid=0 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9f2390fb80 1 client.5826 shutdown 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 msgr2=0x5640b0eca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.946+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000ff70 tx=0x7f9f000047c0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 shutdown_connections 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ea25f0 0x5640b0ee4790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0ea2ed0 0x5640b0eba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9eeffff700 0 client.5826 ms_handle_reset on v2:172.21.15.114:6835/4123842361 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8a490 0x5640b0eca1c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/892812685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0e8d080 0x5640b0ecf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 >> 172.21.15.114:0/892812685 conn(0x5640b0dbb310 msgr2=0x5640b0ed5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 shutdown_connections 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 1 -- 172.21.15.114:0/892812685 wait complete. 2023-11-24T06:26:00.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.947+0000 7f9f2390fb80 20 client.5826 trim_cache size 0 max 16384 2023-11-24T06:26:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 Processor -- start 2023-11-24T06:26:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 -- start start 2023-11-24T06:26:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0ed1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0eb9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0c5f8d0 con 0x5640b0e8d080 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0ed4d80 con 0x5640b0e8a490 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640b0ed4360 con 0x5640b0ea2ed0 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41022/0 (socket says 172.21.15.114:41022) 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.965+0000 7f9f14cb1700 1 -- 172.21.15.114:0/4242581112 learned_addr learned my addr 172.21.15.114:0/4242581112 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 -- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 msgr2=0x5640b0eb9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0eb9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 -- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 msgr2=0x5640b0ed1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0ed1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 -- 172.21.15.114:0/4242581112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b0bf2c10 con 0x5640b0e8d080 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005100 tx=0x7f9f0c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5efc22c47adae2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9efb7fe700 1 -- 172.21.15.114:0/4242581112 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c025040 con 0x5640b0e8d080 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9efb7fe700 1 -- 172.21.15.114:0/4242581112 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c026d00 con 0x5640b0e8d080 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9efb7fe700 1 -- 172.21.15.114:0/4242581112 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c003cb0 con 0x5640b0e8d080 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f2390fb80 1 -- 172.21.15.114:0/4242581112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 msgr2=0x5640b0eb9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.966+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005100 tx=0x7f9f0c00c000 comp rx=0 tx=0).stop 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 -- 172.21.15.114:0/4242581112 shutdown_connections 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0eb9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0ed1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/4242581112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0eb9770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.972 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 -- 172.21.15.114:0/4242581112 >> 172.21.15.114:0/4242581112 conn(0x5640b0dbb310 msgr2=0x5640b0ece400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 -- 172.21.15.114:0/4242581112 shutdown_connections 2023-11-24T06:26:00.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 -- 172.21.15.114:0/4242581112 wait complete. 2023-11-24T06:26:00.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 -- start start 2023-11-24T06:26:00.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.967+0000 7f9f2390fb80 1 Processor -- start 2023-11-24T06:26:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0f2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0ecb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0eea600 con 0x5640b0e8d080 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b0f2fd80 con 0x5640b0e8a490 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640b0f31e20 con 0x5640b0ea2ed0 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f2390fb80 10 client.0 _ll_register_callbacks cb 0x5640b0cc5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41028/0 (socket says 172.21.15.114:41028) 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 -- 172.21.15.114:0/3730699910 learned_addr learned my addr 172.21.15.114:0/3730699910 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 -- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 msgr2=0x5640b0ecb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0ecb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 -- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 msgr2=0x5640b0f2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0f2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.968+0000 7f9f14cb1700 1 -- 172.21.15.114:0/3730699910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640b0c5d5e0 con 0x5640b0e8d080 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f14cb1700 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9f100af950 tx=0x7f9f0c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891d0e3d00b1acd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c025050 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9f0c026440 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f0c006bd0 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f0c026600 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 10 client.5832 fetch_fsmap learned FSMap version 27 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5640b0c30200 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c006bd0 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f0c026760 con 0x5640b0e8d080 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 10 client.5832 fetch_fsmap finished waiting for FSMap version 27 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 10 client.5832 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 20 client.5832 populate_metadata read hostname 'smithi114' 2023-11-24T06:26:00.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ecfce0 0x5640b0ed00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 4 client.5832 mds_command: new command op to 5646 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 --> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5640b0da6f80 con 0x5640b0ecfce0 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.969+0000 7f9f154b2700 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ecfce0 0x5640b0ed00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9eeffff700 10 client.5832 ms_handle_connect on v2:172.21.15.114:6835/4123842361 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f154b2700 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ecfce0 0x5640b0ed00c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c5beb539c31dc0 server_cookie=ff7e237ee6a417eb in_seq=0 out_seq=0 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9eeffff700 1 -- 172.21.15.114:0/3730699910 <== mds.0 v2:172.21.15.114:6835/4123842361 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x5640b0da6f80 con 0x5640b0ecfce0 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9eeffff700 10 client.5832 handle_command_reply: tid=0 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 client.5832 shutdown 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 msgr2=0x5640b0ecb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9f100af950 tx=0x7f9f0c003780 comp rx=0 tx=0).stop 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 shutdown_connections 2023-11-24T06:26:00.979 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640b0ea2ed0 0x5640b0ecb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:6835/4123842361,v1:172.21.15.114:6837/4123842361] conn(0x5640b0ecfce0 0x5640b0ed00c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640b0e8a490 0x5640b0f2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9eeffff700 0 client.5832 ms_handle_reset on v2:172.21.15.114:6835/4123842361 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 --2- 172.21.15.114:0/3730699910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640b0e8d080 0x5640b0ecb490 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 >> 172.21.15.114:0/3730699910 conn(0x5640b0dbb310 msgr2=0x5640b0ed2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 shutdown_connections 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 1 -- 172.21.15.114:0/3730699910 wait complete. 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.970+0000 7f9f2390fb80 20 client.5832 trim_cache size 0 max 16384 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:26:00.980 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "cfb0acb2-02f2-43f9-bba4-c35657dcf6fd", 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 5646, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 10.435615768, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 27, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 47, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 45, 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 15.396535554 2023-11-24T06:26:00.981 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 msgr2=0x7f9ef406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 0x7f9ef406d540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000b9b0 tx=0x7f9f00008040 comp rx=0 tx=0).stop 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 msgr2=0x7f9f100cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f08004000 tx=0x7f9f08004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 shutdown_connections 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9ef406b080 0x7f9ef406d540 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f1015e080 0x7f9f100cc500 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f1006c710 0x7f9f1014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 --2- 172.21.15.114:0/497908869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f100af4d0 0x7f9f101f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:00.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 >> 172.21.15.114:0/497908869 conn(0x7f9f10071b90 msgr2=0x7f9f100b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:00.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 shutdown_connections 2023-11-24T06:26:00.984 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:00.971+0000 7f9f17716700 1 -- 172.21.15.114:0/497908869 wait complete. 2023-11-24T06:26:00.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:26:01.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.104+0000 7f2d2ab51700 1 Processor -- start 2023-11-24T06:26:01.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.104+0000 7f2d2ab51700 1 -- start start 2023-11-24T06:26:01.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d2ab51700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d2406c710 0x7f2d240b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d2ab51700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240ae430 con 0x7f2d2415e080 2023-11-24T06:26:01.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d2ab51700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240ae710 con 0x7f2d240af4d0 2023-11-24T06:26:01.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d2ab51700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240af0b0 con 0x7f2d2406c710 2023-11-24T06:26:01.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d288ed700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2d2415e080 0x7f2d2406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56742/0 (socket says 172.21.15.114:56742) 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d288ed700 1 -- 172.21.15.114:0/4215746131 learned_addr learned my addr 172.21.15.114:0/4215746131 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839400682 0 0) 0x7f2d240ae430 con 0x7f2d2415e080 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d140035b0 con 0x7f2d2415e080 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d23fff700 1 --2- 172.21.15.114:0/4215746131 >> v2:172.21.15.172:3300/0 conn(0x7f2d2406c710 0x7f2d240b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998962970 0 0) 0x7f2d140035b0 con 0x7f2d2415e080 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 >> v2:172.21.15.172:3300/0 conn(0x7f2d2406c710 msgr2=0x7f2d240b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 --2- 172.21.15.114:0/4215746131 >> v2:172.21.15.172:3300/0 conn(0x7f2d2406c710 0x7f2d240b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 >> v1:172.21.15.172:6789/0 conn(0x7f2d240af4d0 legacy=0x7f2d2406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240ae9f0 con 0x7f2d2415e080 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f2d18001670 con 0x7f2d2415e080 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 >> v1:172.21.15.114:6789/0 conn(0x7f2d2415e080 legacy=0x7f2d2406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d14003a00 0x7f2d14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 0x7f2d14004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14005210 0x7f2d1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240af0b0 con 0x7f2d14004850 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400c0b0 con 0x7f2d14005210 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.105+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400c270 con 0x7f2d14003a00 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 0x7f2d14004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d288ed700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d14003a00 0x7f2d14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d290ee700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14005210 0x7f2d1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 -- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d14003a00 msgr2=0x7f2d14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d14003a00 0x7f2d14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 -- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14005210 msgr2=0x7f2d1400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14005210 0x7f2d1400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d290ee700 1 -- 172.21.15.114:0/4215746131 reap_dead start 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 -- 172.21.15.114:0/4215746131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140035b0 con 0x7f2d14004850 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d23fff700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 0x7f2d14004cd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009b60 tx=0x7f2d14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196a17b122cb0988 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000e6c0 con 0x7f2d14004850 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000e820 con 0x7f2d14004850 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.106+0000 7f2d237fe700 1 -- 172.21.15.114:0/4215746131 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000eb90 con 0x7f2d14004850 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 -- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 msgr2=0x7f2d14004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 0x7f2d14004cd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009b60 tx=0x7f2d14005880 comp rx=0 tx=0).stop 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 -- 172.21.15.114:0/4215746131 shutdown_connections 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/4215746131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d14004850 0x7f2d14004cd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 -- 172.21.15.114:0/4215746131 >> 172.21.15.114:0/4215746131 conn(0x7f2d24071b90 msgr2=0x7f2d24071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 -- 172.21.15.114:0/4215746131 shutdown_connections 2023-11-24T06:26:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.107+0000 7f2d2ab51700 1 -- 172.21.15.114:0/4215746131 wait complete. 2023-11-24T06:26:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.108+0000 7f2d2ab51700 1 Processor -- start 2023-11-24T06:26:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.108+0000 7f2d2ab51700 1 -- start start 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14004850 0x7f2d241f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d2414f500 0x7f2d2415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24154620 con 0x7f2d2414f980 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24153cf0 con 0x7f2d14004850 2023-11-24T06:26:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d2ab51700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24153fd0 con 0x7f2d2414f500 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41050/0 (socket says 172.21.15.114:41050) 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 -- 172.21.15.114:0/1108613674 learned_addr learned my addr 172.21.15.114:0/1108613674 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d288ed700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14004850 0x7f2d241f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d23fff700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d2414f500 0x7f2d2415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 -- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d2414f500 msgr2=0x7f2d2415ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d2414f500 0x7f2d2415ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 -- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14004850 msgr2=0x7f2d241f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14004850 0x7f2d241f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.109+0000 7f2d290ee700 1 -- 172.21.15.114:0/1108613674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d241542b0 con 0x7f2d2414f980 2023-11-24T06:26:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d290ee700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2d2406d350 tx=0x7f2d1c00fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003cd0 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c003e30 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c005310 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d24156340 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.110+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241f36e0 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.111+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d240753f0 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.111+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2d1c005470 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.112+0000 7f2d217fa700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 0x7f2d0806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.112+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d1c0898b0 con 0x7f2d2414f980 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.112+0000 7f2d288ed700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 0x7f2d0806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.112+0000 7f2d288ed700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 0x7f2d0806d540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d180016f0 tx=0x7f2d18007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:01.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.116+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d1c0546a0 con 0x7f2d2414f980 2023-11-24T06:26:01.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.348+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d24067f60 con 0x7f2d2414f980 2023-11-24T06:26:01.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.348+0000 7f2d217fa700 1 -- 172.21.15.114:0/1108613674 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d1c057cc0 con 0x7f2d2414f980 2023-11-24T06:26:01.352 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 msgr2=0x7f2d0806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 0x7f2d0806d540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d180016f0 tx=0x7f2d18007e50 comp rx=0 tx=0).stop 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 msgr2=0x7f2d24160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2d2406d350 tx=0x7f2d1c00fe70 comp rx=0 tx=0).stop 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 shutdown_connections 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2d0806b080 0x7f2d0806d540 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d2414f980 0x7f2d24160440 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14004850 0x7f2d241f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 --2- 172.21.15.114:0/1108613674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d2414f500 0x7f2d2415ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 >> 172.21.15.114:0/1108613674 conn(0x7f2d24071b90 msgr2=0x7f2d240b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 shutdown_connections 2023-11-24T06:26:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:01.350+0000 7f2d2ab51700 1 -- 172.21.15.114:0/1108613674 wait complete. 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:26:01.366 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:26:01.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.367 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2023-11-24T06:26:01.384 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:01.385 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42293sec preferred_lft 42293sec 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::9ca9:e760:b5c4:9eac/64 scope link noprefixroute 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.386 INFO:teuthology.orchestra.run.smithi114.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::385c:94ff:fec5:c8f4/64 scope link 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether fa:2a:57:62:51:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::f82a:57ff:fe62:518a/64 scope link 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:26:01.387 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:26:01.388 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::acf4:a2ff:fe10:15ae/64 scope link 2023-11-24T06:26:01.388 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:01.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.388 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:26:01.450 INFO:teuthology.orchestra.run.smithi114.stdout:1 2023-11-24T06:26:01.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.451 DEBUG:teuthology.orchestra.run.smithi114:> route 2023-11-24T06:26:01.510 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2023-11-24T06:26:01.510 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:26:01.510 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:26:01.510 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:26:01.511 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:26:01.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:26:01.511 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:26:01.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.614 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:26:01.632 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:26:01.632 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:26:01.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:26:01.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.633 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2023-11-24T06:26:01.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:26:01.699 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:26:01.756 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:26:01.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:26:01.756 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:26:01.812 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:26:01.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:01.813 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:01.898 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:01.898 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:01.921 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:01.921 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:01.921 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:26:01.922 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:26:01.922 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:26:01.922 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:26:01.922 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:26:01.922 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:26:02.003 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:26:02.004 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:26:02.004 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:26:02.004 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:26:02.004 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:26:02.005 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:26:02.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:26:02.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.033 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:26:02.117 INFO:teuthology.orchestra.run.smithi114.stdout:client5081 2023-11-24T06:26:02.117 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:26:02.118 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:02.176 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:26:02.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.332+0000 7f5d34417700 1 Processor -- start 2023-11-24T06:26:02.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.332+0000 7f5d34417700 1 -- start start 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.333+0000 7f5d34417700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5d2c158670 0x7f5d2c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.333+0000 7f5d34417700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0cfff0 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.333+0000 7f5d34417700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0cfd10 con 0x7f5d2c158a50 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.333+0000 7f5d34417700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c161f50 con 0x7f5d2c158670 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d329b4700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5d2c0c9fc0 0x7f5d2c0ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56770/0 (socket says 172.21.15.114:56770) 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d329b4700 1 -- 172.21.15.114:0/2016937163 learned_addr learned my addr 172.21.15.114:0/2016937163 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d321b3700 1 --2- 172.21.15.114:0/2016937163 >> v2:172.21.15.172:3300/0 conn(0x7f5d2c158670 0x7f5d2c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843544322 0 0) 0x7f5d2c0cfff0 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d180035b0 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659177725 0 0) 0x7f5d180035b0 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 >> v2:172.21.15.172:3300/0 conn(0x7f5d2c158670 msgr2=0x7f5d2c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 --2- 172.21.15.114:0/2016937163 >> v2:172.21.15.172:3300/0 conn(0x7f5d2c158670 0x7f5d2c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 >> v1:172.21.15.172:6789/0 conn(0x7f5d2c158a50 legacy=0x7f5d2c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c162230 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f5d240019f0 con 0x7f5d2c0c9fc0 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 >> v1:172.21.15.114:6789/0 conn(0x7f5d2c0c9fc0 legacy=0x7f5d2c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d18003a00 0x7f5d18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d18004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d18005330 0x7f5d1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c161f50 con 0x7f5d18004850 2023-11-24T06:26:02.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1800c200 con 0x7f5d18003a00 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1800c3c0 con 0x7f5d18005330 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d319b2700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d18004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d321b3700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d18003a00 0x7f5d18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.334+0000 7f5d329b4700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d18005330 0x7f5d1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 -- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d18005330 msgr2=0x7f5d1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d18005330 0x7f5d1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 -- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d18003a00 msgr2=0x7f5d18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d18003a00 0x7f5d18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 -- 172.21.15.114:0/2016937163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d180035b0 con 0x7f5d18004850 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d329b4700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d18005330 0x7f5d1800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d329b4700 1 -- 172.21.15.114:0/2016937163 reap_dead start 2023-11-24T06:26:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d319b2700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d18004cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d28007fd0 tx=0x7f5d18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9232b41c36ceba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280052d0 con 0x7f5d18004850 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28005430 con 0x7f5d18004850 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d311b1700 1 -- 172.21.15.114:0/2016937163 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280038c0 con 0x7f5d18004850 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 -- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 msgr2=0x7f5d18004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d18004cd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d28007fd0 tx=0x7f5d18005880 comp rx=0 tx=0).stop 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 -- 172.21.15.114:0/2016937163 shutdown_connections 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 --2- 172.21.15.114:0/2016937163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d18004cd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 -- 172.21.15.114:0/2016937163 >> 172.21.15.114:0/2016937163 conn(0x7f5d2c0b57b0 msgr2=0x7f5d2c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.335+0000 7f5d34417700 1 -- 172.21.15.114:0/2016937163 shutdown_connections 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.336+0000 7f5d34417700 1 -- 172.21.15.114:0/2016937163 wait complete. 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 Processor -- start 2023-11-24T06:26:02.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 -- start start 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d2c15f8a0 0x7f5d2c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 0x7f5d2c160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c149c80 con 0x7f5d18004850 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b4310 con 0x7f5d2c15f8a0 2023-11-24T06:26:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d34417700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0ae040 con 0x7f5d2c0b3e90 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d321b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d321b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41070/0 (socket says 172.21.15.114:41070) 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.337+0000 7f5d321b3700 1 -- 172.21.15.114:0/4094621516 learned_addr learned my addr 172.21.15.114:0/4094621516 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d319b2700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d2c15f8a0 0x7f5d2c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d329b4700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 0x7f5d2c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 -- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 msgr2=0x7f5d2c160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 0x7f5d2c160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 -- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d2c15f8a0 msgr2=0x7f5d2c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d2c15f8a0 0x7f5d2c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 -- 172.21.15.114:0/4094621516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0ae320 con 0x7f5d18004850 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d329b4700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 0x7f5d2c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:02.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d321b3700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c011aa0 tx=0x7f5d1c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f8e0 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00fa40 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00fdf0 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0ae600 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.338+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c0aebc0 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.339+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c067f60 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.339+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5d1c014440 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.340+0000 7f5d22ffd700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 0x7f5d1006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.340+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5d1c08a0f0 con 0x7f5d18004850 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.340+0000 7f5d319b2700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 0x7f5d1006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.340+0000 7f5d319b2700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 0x7f5d1006d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d28007fd0 tx=0x7f5d28009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.344+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d1c054f70 con 0x7f5d18004850 2023-11-24T06:26:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.581+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d2c0cb3c0 con 0x7f5d18004850 2023-11-24T06:26:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.581+0000 7f5d22ffd700 1 -- 172.21.15.114:0/4094621516 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d1c058590 con 0x7f5d18004850 2023-11-24T06:26:02.585 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 msgr2=0x7f5d1006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 0x7f5d1006d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d28007fd0 tx=0x7f5d28009030 comp rx=0 tx=0).stop 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 msgr2=0x7f5d2c149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c011aa0 tx=0x7f5d1c011db0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 shutdown_connections 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5d1006b080 0x7f5d1006d540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d2c0b3e90 0x7f5d2c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d18004850 0x7f5d2c149740 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 --2- 172.21.15.114:0/4094621516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d2c15f8a0 0x7f5d2c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 >> 172.21.15.114:0/4094621516 conn(0x7f5d2c0b57b0 msgr2=0x7f5d2c0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 shutdown_connections 2023-11-24T06:26:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:02.583+0000 7f5d34417700 1 -- 172.21.15.114:0/4094621516 wait complete. 2023-11-24T06:26:02.598 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:26:02.599 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:26:02.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.600 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-11-24T06:26:02.621 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:26:02.621 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:26:02.622 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c36d:1855:677d:d218/64 scope link noprefixroute 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::4cb3:adff:fe77:17e6/64 scope link 2023-11-24T06:26:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::3838:94ff:fea3:1687/64 scope link 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 56:7e:24:58:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::547e:24ff:fe58:9cd/64 scope link 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:26:02.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.625 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:26:02.691 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:26:02.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.693 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-11-24T06:26:02.754 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-11-24T06:26:02.755 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:26:02.755 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:26:02.755 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:26:02.755 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:26:02.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.756 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:26:02.756 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:26:02.756 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:26:02.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:26:02.757 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:26:02.757 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:26:02.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.863 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:26:02.882 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:26:02.882 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:26:02.882 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:26:02.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:02.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-11-24T06:26:02.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:26:02.949 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:26:03.006 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:26:03.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:26:03.007 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:26:03.063 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:26:03.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:03.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:03.150 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:03.150 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:03.178 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:03.178 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:03.178 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:26:03.178 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:26:03.179 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:26:03.179 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:26:03.179 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:26:03.179 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:26:03.247 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:26:03.248 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:26:03.248 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:26:03.248 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:26:03.249 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:26:03.249 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:26:03.249 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:26:03.249 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:26:03.250 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:26:03.250 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:26:03.250 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:26:03.251 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:26:03.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:03.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:26:03.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:03.278 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:26:03.351 INFO:teuthology.orchestra.run.smithi172.stdout:client5359 2023-11-24T06:26:03.351 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:26:03.351 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:03.408 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:26:03.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 Processor -- start 2023-11-24T06:26:03.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 -- start start 2023-11-24T06:26:03.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29a40cf040 0x7f29a406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40d21c0 0x7f29a40af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a40d01c0 con 0x7f29a40702e0 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a4070910 con 0x7f29a40cf040 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.543+0000 7f29a9cd6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4070bf0 con 0x7f29a40d21c0 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29a40cf040 0x7f29a406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29a40cf040 0x7f29a406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41078/0 (socket says 172.21.15.114:41078) 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 -- 172.21.15.114:0/3116732904 learned_addr learned my addr 172.21.15.114:0/3116732904 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a3fff700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40d21c0 0x7f29a40af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 -- 172.21.15.114:0/3116732904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40d21c0 msgr2=0x7f29a40af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40d21c0 0x7f29a40af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 -- 172.21.15.114:0/3116732904 >> v1:172.21.15.172:6789/0 conn(0x7f29a40702e0 legacy=0x7f29a40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 -- 172.21.15.114:0/3116732904 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40aebb0 con 0x7f29a40cf040 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a37fe700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40cf040 0x7f29a406fda0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f29940098f0 tx=0x7f2994009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53ab8c60f8ca69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a27fc700 1 -- 172.21.15.114:0/3116732904 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994011830 con 0x7f29a40cf040 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a27fc700 1 -- 172.21.15.114:0/3116732904 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2994011990 con 0x7f29a40cf040 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.544+0000 7f29a27fc700 1 -- 172.21.15.114:0/3116732904 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994011d20 con 0x7f29a40cf040 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3116732904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40cf040 msgr2=0x7f29a406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40cf040 0x7f29a406fda0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f29940098f0 tx=0x7f2994009c00 comp rx=0 tx=0).stop 2023-11-24T06:26:03.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3116732904 shutdown_connections 2023-11-24T06:26:03.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40d21c0 0x7f29a40af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3116732904 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40cf040 0x7f29a406fda0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3116732904 >> 172.21.15.114:0/3116732904 conn(0x7f29a409aa00 msgr2=0x7f29a409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:03.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3116732904 shutdown_connections 2023-11-24T06:26:03.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.545+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3116732904 wait complete. 2023-11-24T06:26:03.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.546+0000 7f29a9cd6700 1 Processor -- start 2023-11-24T06:26:03.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.546+0000 7f29a9cd6700 1 -- start start 2023-11-24T06:26:03.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 0x7f29a40c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 0x7f29a40c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40ca810 con 0x7f29a40702e0 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a41571e0 con 0x7f29a40d21c0 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a9cd6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a41576c0 con 0x7f29a40cf040 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41084/0 (socket says 172.21.15.114:41084) 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 -- 172.21.15.114:0/3985655774 learned_addr learned my addr 172.21.15.114:0/3985655774 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a3fff700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 0x7f29a40c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a2ffd700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 0x7f29a40c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 -- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 msgr2=0x7f29a40c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 0x7f29a40c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 -- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 msgr2=0x7f29a40c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 0x7f29a40c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 -- 172.21.15.114:0/3985655774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994009550 con 0x7f29a40702e0 2023-11-24T06:26:03.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a2ffd700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 0x7f29a40c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a3fff700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 0x7f29a40c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.547+0000 7f29a37fe700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.548+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994004570 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.548+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a40ca250 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.548+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a41f3540 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.548+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299401f070 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.548+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940038c0 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.549+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29880052e0 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.549+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2994021030 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.550+0000 7f299bfff700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 0x7f298406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.550+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f299408ad80 con 0x7f29a40702e0 2023-11-24T06:26:03.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.550+0000 7f29a2ffd700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 0x7f298406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:03.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.550+0000 7f29a2ffd700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 0x7f298406d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29a40b4600 tx=0x7f298c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:03.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.554+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2994055ca0 con 0x7f29a40702e0 2023-11-24T06:26:03.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.790+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2988005860 con 0x7f29a40702e0 2023-11-24T06:26:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.877+0000 7f299bfff700 1 -- 172.21.15.114:0/3985655774 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 136+0+0 (secure 0 0 0) 0x7f29940150c0 con 0x7f29a40702e0 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 msgr2=0x7f298406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 0x7f298406d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29a40b4600 tx=0x7f298c009040 comp rx=0 tx=0).stop 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 msgr2=0x7f29a4156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994004750 comp rx=0 tx=0).stop 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 shutdown_connections 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f298406b080 0x7f298406d540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29a40d21c0 0x7f29a40c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29a40702e0 0x7f29a4156ca0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 --2- 172.21.15.114:0/3985655774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29a40cf040 0x7f29a40c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 >> 172.21.15.114:0/3985655774 conn(0x7f29a409aa00 msgr2=0x7f29a406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 shutdown_connections 2023-11-24T06:26:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:03.879+0000 7f29a9cd6700 1 -- 172.21.15.114:0/3985655774 wait complete. 2023-11-24T06:26:03.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:26:04.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.012+0000 7f0eba7a9700 1 Processor -- start 2023-11-24T06:26:04.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.012+0000 7f0eba7a9700 1 -- start start 2023-11-24T06:26:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eba7a9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0eb41446a0 0x7f0eb40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eba7a9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0eb40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eba7a9700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40af950 con 0x7f0eb40adf00 2023-11-24T06:26:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eba7a9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb4144a80 con 0x7f0eb41446a0 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eba7a9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40afc30 con 0x7f0eb406c710 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eb37fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0eb40adf00 0x7f0eb406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56800/0 (socket says 172.21.15.114:56800) 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eb37fe700 1 -- 172.21.15.114:0/1732104266 learned_addr learned my addr 172.21.15.114:0/1732104266 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0eb40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/1732104266 >> v2:172.21.15.172:3300/0 conn(0x7f0eb41446a0 0x7f0eb40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.013+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211580937 0 0) 0x7f0eb40af950 con 0x7f0eb40adf00 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e9c0035b0 con 0x7f0eb40adf00 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776022972 0 0) 0x7f0e9c0035b0 con 0x7f0eb40adf00 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 msgr2=0x7f0eb40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0eb40b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 >> v2:172.21.15.172:3300/0 conn(0x7f0eb41446a0 msgr2=0x7f0eb40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 --2- 172.21.15.114:0/1732104266 >> v2:172.21.15.172:3300/0 conn(0x7f0eb41446a0 0x7f0eb40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40aff10 con 0x7f0eb40adf00 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0ea4001670 con 0x7f0eb40adf00 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 >> v1:172.21.15.114:6789/0 conn(0x7f0eb40adf00 legacy=0x7f0eb406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e9c003a00 0x7f0e9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0e9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4144a80 con 0x7f0e9c004d90 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40afc30 con 0x7f0e9c003a00 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c00ba10 con 0x7f0eb406c710 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb37fe700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0e9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e9c003a00 0x7f0e9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 -- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 msgr2=0x7f0e9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406c710 0x7f0e9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 -- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e9c003a00 msgr2=0x7f0e9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e9c003a00 0x7f0e9c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 -- 172.21.15.114:0/1732104266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0035b0 con 0x7f0e9c004d90 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.014+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fc0 tx=0x7f0e9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9edf27721f0d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ec30 con 0x7f0e9c004d90 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00f040 con 0x7f0e9c004d90 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eb2ffd700 1 -- 172.21.15.114:0/1732104266 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac014480 con 0x7f0e9c004d90 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1732104266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 msgr2=0x7f0e9c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1732104266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0e9c005210 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009fc0 tx=0x7f0e9c005880 comp rx=0 tx=0).stop 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eb8d46700 1 -- 172.21.15.114:0/1732104266 reap_dead start 2023-11-24T06:26:04.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1732104266 shutdown_connections 2023-11-24T06:26:04.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1732104266 >> 172.21.15.114:0/1732104266 conn(0x7f0eb4071b90 msgr2=0x7f0eb4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:04.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1732104266 shutdown_connections 2023-11-24T06:26:04.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.015+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1732104266 wait complete. 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.016+0000 7f0eba7a9700 1 Processor -- start 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 -- start start 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 0x7f0eb409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 0x7f0eb409af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb409e490 con 0x7f0e9c004d90 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41563d0 con 0x7f0eb4156600 2023-11-24T06:26:04.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eba7a9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b0f60 con 0x7f0eb406e880 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eb3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eb3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41104/0 (socket says 172.21.15.114:41104) 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.017+0000 7f0eb3fff700 1 -- 172.21.15.114:0/1545126461 learned_addr learned my addr 172.21.15.114:0/1545126461 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb37fe700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 0x7f0eb409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 0x7f0eb409af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 -- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 msgr2=0x7f0eb409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 0x7f0eb409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 -- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 msgr2=0x7f0eb409af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 0x7f0eb409af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 -- 172.21.15.114:0/1545126461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb409e730 con 0x7f0e9c004d90 2023-11-24T06:26:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb37fe700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 0x7f0eb409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb8d46700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 0x7f0eb409af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011aa0 tx=0x7f0ea8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800f980 con 0x7f0e9c004d90 2023-11-24T06:26:04.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea800fb00 con 0x7f0e9c004d90 2023-11-24T06:26:04.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb40b1240 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800feb0 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.018+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb4156d10 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.019+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb40753f0 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.019+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0ea8014440 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.020+0000 7f0eb0ff9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 0x7f0ea006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.020+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0ea808a0d0 con 0x7f0e9c004d90 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.020+0000 7f0eb37fe700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 0x7f0ea006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.020+0000 7f0eb37fe700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 0x7f0ea006d4d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40d0b80 tx=0x7f0ea4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.024+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ea8055010 con 0x7f0e9c004d90 2023-11-24T06:26:04.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.242+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb40b6020 con 0x7f0ea006b010 2023-11-24T06:26:04.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.243+0000 7f0eb0ff9700 1 -- 172.21.15.114:0/1545126461 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0ea4007120 con 0x7f0ea006b010 2023-11-24T06:26:04.247 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:26:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:26:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephfs" 2023-11-24T06:26:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:26:04.247 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:26:04.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 msgr2=0x7f0ea006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 0x7f0ea006d4d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40d0b80 tx=0x7f0ea4009040 comp rx=0 tx=0).stop 2023-11-24T06:26:04.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 msgr2=0x7f0eb4155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8011aa0 tx=0x7f0ea8011e60 comp rx=0 tx=0).stop 2023-11-24T06:26:04.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 shutdown_connections 2023-11-24T06:26:04.249 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0ea006b010 0x7f0ea006d4d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4156600 0x7f0eb409af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e9c004d90 0x7f0eb4155e90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 --2- 172.21.15.114:0/1545126461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0eb406e880 0x7f0eb409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 >> 172.21.15.114:0/1545126461 conn(0x7f0eb4071b90 msgr2=0x7f0eb40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:04.250 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 shutdown_connections 2023-11-24T06:26:04.251 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.245+0000 7f0eba7a9700 1 -- 172.21.15.114:0/1545126461 wait complete. 2023-11-24T06:26:04.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:26:04.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.380+0000 7f7cd95c6700 1 Processor -- start 2023-11-24T06:26:04.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.380+0000 7f7cd95c6700 1 -- start start 2023-11-24T06:26:04.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd95c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd415e160 0x7f7cd40d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd95c6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd95c6700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd40cfe20 con 0x7f7cd40d20d0 2023-11-24T06:26:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd95c6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd40d2550 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd95c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40b2bb0 con 0x7f7cd415e160 2023-11-24T06:26:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41116/0 (socket says 172.21.15.114:41116) 2023-11-24T06:26:04.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 -- 172.21.15.114:0/2475258963 learned_addr learned my addr 172.21.15.114:0/2475258963 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:04.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd2ffd700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd415e160 0x7f7cd40d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 -- 172.21.15.114:0/2475258963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd415e160 msgr2=0x7f7cd40d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd415e160 0x7f7cd40d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 -- 172.21.15.114:0/2475258963 >> v1:172.21.15.172:6789/0 conn(0x7f7cd40d20d0 legacy=0x7f7cd40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.381+0000 7f7cd37fe700 1 -- 172.21.15.114:0/2475258963 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd415e540 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd37fe700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009e50 tx=0x7f7ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca67f49c343be933 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/2475258963 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00e070 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/2475258963 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc00b9c0 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd1ffb700 1 -- 172.21.15.114:0/2475258963 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00bd70 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 -- 172.21.15.114:0/2475258963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd40b1fd0 msgr2=0x7f7cd406b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009e50 tx=0x7f7ccc00b330 comp rx=0 tx=0).stop 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 -- 172.21.15.114:0/2475258963 shutdown_connections 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd406b4c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/2475258963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd415e160 0x7f7cd40d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 -- 172.21.15.114:0/2475258963 >> 172.21.15.114:0/2475258963 conn(0x7f7cd4073130 msgr2=0x7f7cd409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 -- 172.21.15.114:0/2475258963 shutdown_connections 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.382+0000 7f7cd95c6700 1 -- 172.21.15.114:0/2475258963 wait complete. 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.384+0000 7f7cd95c6700 1 Processor -- start 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.384+0000 7f7cd95c6700 1 -- start start 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd4160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd40d20d0 0x7f7cd40b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40cac20 con 0x7f7cd415e160 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ca010 con 0x7f7cd40b1fd0 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd95c6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ca2f0 con 0x7f7cd40d20d0 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41122/0 (socket says 172.21.15.114:41122) 2023-11-24T06:26:04.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 -- 172.21.15.114:0/1582225540 learned_addr learned my addr 172.21.15.114:0/1582225540 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd2ffd700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd4160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd27fc700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd40d20d0 0x7f7cd40b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 -- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd40d20d0 msgr2=0x7f7cd40b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd40d20d0 0x7f7cd40b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 -- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cd40b1fd0 msgr2=0x7f7cd4160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd4160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.385+0000 7f7cd37fe700 1 -- 172.21.15.114:0/1582225540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc009b30 con 0x7f7cd415e160 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cd37fe700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7cd406eae0 tx=0x7f7ccc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc004840 con 0x7f7cd415e160 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc0099b0 con 0x7f7cd415e160 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd40ca5d0 con 0x7f7cd415e160 2023-11-24T06:26:04.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc01c680 con 0x7f7cd415e160 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.386+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41614c0 con 0x7f7cd415e160 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.387+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4068fb0 con 0x7f7cd415e160 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.387+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f7ccc01c7e0 con 0x7f7cd415e160 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.388+0000 7f7cbb7fe700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 0x7f7cc006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.388+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7ccc08b540 con 0x7f7cd415e160 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.388+0000 7f7cd2ffd700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 0x7f7cc006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.388+0000 7f7cd2ffd700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 0x7f7cc006d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006b20 tx=0x7f7cc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.392+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ccc0562b0 con 0x7f7cd415e160 2023-11-24T06:26:04.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.583+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7cd40b4180 con 0x7f7cd415e160 2023-11-24T06:26:04.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.583+0000 7f7cbb7fe700 1 -- 172.21.15.114:0/1582225540 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ccc0598d0 con 0x7f7cd415e160 2023-11-24T06:26:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 msgr2=0x7f7cc006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 0x7f7cc006d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8006b20 tx=0x7f7cc8008040 comp rx=0 tx=0).stop 2023-11-24T06:26:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 msgr2=0x7f7cd4159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7cd406eae0 tx=0x7f7ccc004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 shutdown_connections 2023-11-24T06:26:04.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7cc006b080 0x7f7cc006d540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cd415e160 0x7f7cd4159760 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.591 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cd40b1fd0 0x7f7cd4160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 --2- 172.21.15.114:0/1582225540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7cd40d20d0 0x7f7cd40b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 >> 172.21.15.114:0/1582225540 conn(0x7f7cd4073130 msgr2=0x7f7cd40d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 shutdown_connections 2023-11-24T06:26:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.585+0000 7f7cd95c6700 1 -- 172.21.15.114:0/1582225540 wait complete. 2023-11-24T06:26:04.600 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:26:04.600 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:04.622 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:26:04.622 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:04.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.838+0000 7fe0b803a700 1 Processor -- start 2023-11-24T06:26:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 -- start start 2023-11-24T06:26:04.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe0b006e0b0 0x7fe0b00b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00b0290 con 0x7fe0b00b5dd0 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00b4820 con 0x7fe0b006e0b0 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.839+0000 7fe0b803a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00b4b00 con 0x7fe0b00b59f0 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe0b006e0b0 0x7fe0b00b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b65d7700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe0b00b5dd0 0x7fe0b00af190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56836/0 (socket says 172.21.15.114:56836) 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b65d7700 1 -- 172.21.15.114:0/3662019189 learned_addr learned my addr 172.21.15.114:0/3662019189 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b5dd6700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 -- 172.21.15.114:0/3662019189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 msgr2=0x7fe0b006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b006db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 -- 172.21.15.114:0/3662019189 >> v1:172.21.15.114:6789/0 conn(0x7fe0b00b5dd0 legacy=0x7fe0b00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 -- 172.21.15.114:0/3662019189 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00ae9b0 con 0x7fe0b006e0b0 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b4dd4700 1 -- 172.21.15.114:0/3662019189 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538141637 0 0) 0x7fe0b00b0290 con 0x7fe0b00b5dd0 2023-11-24T06:26:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b5dd6700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b55d5700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b00b3c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009840 tx=0x7fe0a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e08007f26937ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b4dd4700 1 -- 172.21.15.114:0/3662019189 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011830 con 0x7fe0b006e0b0 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b4dd4700 1 -- 172.21.15.114:0/3662019189 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4011990 con 0x7fe0b006e0b0 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.840+0000 7fe0b4dd4700 1 -- 172.21.15.114:0/3662019189 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011d20 con 0x7fe0b006e0b0 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 -- 172.21.15.114:0/3662019189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 msgr2=0x7fe0b00b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.848 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b00b3c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009840 tx=0x7fe0a4009b50 comp rx=0 tx=0).stop 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 -- 172.21.15.114:0/3662019189 shutdown_connections 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3662019189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b00b3c40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 -- 172.21.15.114:0/3662019189 >> 172.21.15.114:0/3662019189 conn(0x7fe0b0072990 msgr2=0x7fe0b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 -- 172.21.15.114:0/3662019189 shutdown_connections 2023-11-24T06:26:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.841+0000 7fe0b803a700 1 -- 172.21.15.114:0/3662019189 wait complete. 2023-11-24T06:26:04.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.842+0000 7fe0b803a700 1 Processor -- start 2023-11-24T06:26:04.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.842+0000 7fe0b803a700 1 -- start start 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b013ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 0x7fe0b00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014bca0 con 0x7fe0b006e0b0 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014f360 con 0x7fe0b00b5dd0 2023-11-24T06:26:04.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b803a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014f640 con 0x7fe0b00b59f0 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41132/0 (socket says 172.21.15.114:41132) 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 -- 172.21.15.114:0/3612870287 learned_addr learned my addr 172.21.15.114:0/3612870287 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b55d5700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b013ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b65d7700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 0x7fe0b00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 -- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 msgr2=0x7fe0b013ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b013ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 -- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 msgr2=0x7fe0b00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 0x7fe0b00cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 -- 172.21.15.114:0/3612870287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40094a0 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b65d7700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 0x7fe0b00cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0b5dd6700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e810 tx=0x7fe0ac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.843+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac013610 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.844+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac013770 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.844+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b014f920 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.844+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac013b20 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.844+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b014fee0 con 0x7fe0b006e0b0 2023-11-24T06:26:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.845+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b0150250 con 0x7fe0b006e0b0 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.845+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe0ac013c80 con 0x7fe0b006e0b0 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.845+0000 7fe0a27fc700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 0x7fe09406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.845+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0ac089690 con 0x7fe0b006e0b0 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.845+0000 7fe0b55d5700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 0x7fe09406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.846+0000 7fe0b55d5700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 0x7fe09406d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe0b006e4b0 tx=0x7fe0a4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:04.850+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0ac054400 con 0x7fe0b006e0b0 2023-11-24T06:26:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.044+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0b0067f60 con 0x7fe0b006e0b0 2023-11-24T06:26:05.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.045+0000 7fe0a27fc700 1 -- 172.21.15.114:0/3612870287 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fe0ac057a20 con 0x7fe0b006e0b0 2023-11-24T06:26:05.049 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:05.049 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:59.863101+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:05.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 msgr2=0x7fe09406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 0x7fe09406d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe0b006e4b0 tx=0x7fe0a4008030 comp rx=0 tx=0).stop 2023-11-24T06:26:05.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 msgr2=0x7fe0b013e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e810 tx=0x7fe0ac00ebd0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 shutdown_connections 2023-11-24T06:26:05.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe09406b080 0x7fe09406d540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.052 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0b00b5dd0 0x7fe0b00cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0b006e0b0 0x7fe0b013e3e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 --2- 172.21.15.114:0/3612870287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0b00b59f0 0x7fe0b013ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 >> 172.21.15.114:0/3612870287 conn(0x7fe0b0072990 msgr2=0x7fe0b0158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 shutdown_connections 2023-11-24T06:26:05.053 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.047+0000 7fe0b803a700 1 -- 172.21.15.114:0/3612870287 wait complete. 2023-11-24T06:26:05.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:48.353169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:48.873132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:26:05.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:26:05.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.183+0000 7f77aaedc700 1 Processor -- start 2023-11-24T06:26:05.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.183+0000 7f77aaedc700 1 -- start start 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.183+0000 7f77aaedc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f77a415bf50 0x7f77a40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77aaedc700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40d0140 con 0x7f77a40c6c30 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77aaedc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40cf3c0 con 0x7f77a40ce830 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77aaedc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a415c330 con 0x7f77a415bf50 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a8c78700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f77a415bf50 0x7f77a40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a9479700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f77a40c6c30 0x7f77a40cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56852/0 (socket says 172.21.15.114:56852) 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a9479700 1 -- 172.21.15.114:0/2989824457 learned_addr learned my addr 172.21.15.114:0/2989824457 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a8c78700 1 -- 172.21.15.114:0/2989824457 >> v1:172.21.15.172:6789/0 conn(0x7f77a40ce830 legacy=0x7f77a40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a8c78700 1 -- 172.21.15.114:0/2989824457 >> v1:172.21.15.114:6789/0 conn(0x7f77a40c6c30 legacy=0x7f77a40cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a37fe700 1 -- 172.21.15.114:0/2989824457 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011899911 0 0) 0x7f77a40d0140 con 0x7f77a40c6c30 2023-11-24T06:26:05.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a8c78700 1 -- 172.21.15.114:0/2989824457 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40d03e0 con 0x7f77a415bf50 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a8c78700 1 --2- 172.21.15.114:0/2989824457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a415bf50 0x7f77a40ce2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7798009840 tx=0x7f7798009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae8d67e6d132682 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a37fe700 1 -- 172.21.15.114:0/2989824457 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798004230 con 0x7f77a415bf50 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a37fe700 1 -- 172.21.15.114:0/2989824457 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798004390 con 0x7f77a415bf50 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.184+0000 7f77a37fe700 1 -- 172.21.15.114:0/2989824457 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980036a0 con 0x7f77a415bf50 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 -- 172.21.15.114:0/2989824457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a415bf50 msgr2=0x7f77a40ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 --2- 172.21.15.114:0/2989824457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a415bf50 0x7f77a40ce2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7798009840 tx=0x7f7798009b50 comp rx=0 tx=0).stop 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 -- 172.21.15.114:0/2989824457 shutdown_connections 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 --2- 172.21.15.114:0/2989824457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a415bf50 0x7f77a40ce2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 -- 172.21.15.114:0/2989824457 >> 172.21.15.114:0/2989824457 conn(0x7f77a40b3780 msgr2=0x7f77a40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 -- 172.21.15.114:0/2989824457 shutdown_connections 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.185+0000 7f77aaedc700 1 -- 172.21.15.114:0/2989824457 wait complete. 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.186+0000 7f77aaedc700 1 Processor -- start 2023-11-24T06:26:05.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.186+0000 7f77aaedc700 1 -- start start 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a40c6c30 0x7f77a406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a40ce830 0x7f77a41e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40ca1f0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41555a0 con 0x7f77a40ce830 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77aaedc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41e68d0 con 0x7f77a40c6c30 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41150/0 (socket says 172.21.15.114:41150) 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 -- 172.21.15.114:0/369710742 learned_addr learned my addr 172.21.15.114:0/369710742 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a3fff700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a40ce830 0x7f77a41e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a8c78700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a40c6c30 0x7f77a406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 -- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a40c6c30 msgr2=0x7f77a406cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a40c6c30 0x7f77a406cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 -- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a40ce830 msgr2=0x7f77a41e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a40ce830 0x7f77a41e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.187+0000 7f77a9479700 1 -- 172.21.15.114:0/369710742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980094a0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77a9479700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f779c00cc10 tx=0x7f779c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00aa60 con 0x7f77a41f3ee0 2023-11-24T06:26:05.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00abc0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a41e65f0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c0070d0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.188+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a40ca700 con 0x7f77a41f3ee0 2023-11-24T06:26:05.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.189+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a40cf230 con 0x7f77a41f3ee0 2023-11-24T06:26:05.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.189+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f779c007230 con 0x7f77a41f3ee0 2023-11-24T06:26:05.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.190+0000 7f77a17fa700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 0x7f778806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.190+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f779c089690 con 0x7f77a41f3ee0 2023-11-24T06:26:05.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.190+0000 7f77a8c78700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 0x7f778806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.190+0000 7f77a8c78700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 0x7f778806d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77a4071c50 tx=0x7f779801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.194+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f779c054480 con 0x7f77a41f3ee0 2023-11-24T06:26:05.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.382+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77a4068fb0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.382+0000 7f77a17fa700 1 -- 172.21.15.114:0/369710742 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f779c057aa0 con 0x7f77a41f3ee0 2023-11-24T06:26:05.386 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:26:05.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 msgr2=0x7f778806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 0x7f778806d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f77a4071c50 tx=0x7f779801c040 comp rx=0 tx=0).stop 2023-11-24T06:26:05.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 msgr2=0x7f77a4154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f779c00cc10 tx=0x7f779c00cf20 comp rx=0 tx=0).stop 2023-11-24T06:26:05.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 shutdown_connections 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.384+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f778806b080 0x7f778806d540 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77a41f3ee0 0x7f77a4154970 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a40c6c30 0x7f77a406cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 --2- 172.21.15.114:0/369710742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a40ce830 0x7f77a41e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 >> 172.21.15.114:0/369710742 conn(0x7f77a40b3780 msgr2=0x7f77a40b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 shutdown_connections 2023-11-24T06:26:05.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.385+0000 7f77aaedc700 1 -- 172.21.15.114:0/369710742 wait complete. 2023-11-24T06:26:05.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:26:05.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.520+0000 7f3d87bf9700 1 Processor -- start 2023-11-24T06:26:05.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.520+0000 7f3d87bf9700 1 -- start start 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d87bf9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3d8015b370 0x7f3d8015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d87bf9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d800b3c60 0x7f3d800b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d87bf9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d800ae5e0 con 0x7f3d801587f0 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d87bf9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d800b4630 con 0x7f3d8015b370 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d87bf9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8015be30 con 0x7f3d800b3c60 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3d8015b370 0x7f3d8015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d86196700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d800b3c60 0x7f3d800b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85995700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3d801587f0 0x7f3d8006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:50458/0 (socket says 172.21.15.114:50458) 2023-11-24T06:26:05.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85995700 1 -- 172.21.15.114:0/2085854536 learned_addr learned my addr 172.21.15.114:0/2085854536 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:05.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 -- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d800b3c60 msgr2=0x7f3d800b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:05.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 --2- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d800b3c60 0x7f3d800b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 -- 172.21.15.114:0/2085854536 >> v1:172.21.15.172:6789/0 conn(0x7f3d801587f0 legacy=0x7f3d8006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d84993700 1 -- 172.21.15.114:0/2085854536 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526236164 0 0) 0x7f3d800ae5e0 con 0x7f3d801587f0 2023-11-24T06:26:05.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 -- 172.21.15.114:0/2085854536 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8015c110 con 0x7f3d8015b370 2023-11-24T06:26:05.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.521+0000 7f3d85194700 1 --2- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d8015b370 0x7f3d8015b800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d70009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6605cf1076a4fd2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d84993700 1 -- 172.21.15.114:0/2085854536 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70011830 con 0x7f3d8015b370 2023-11-24T06:26:05.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d84993700 1 -- 172.21.15.114:0/2085854536 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d70011990 con 0x7f3d8015b370 2023-11-24T06:26:05.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d84993700 1 -- 172.21.15.114:0/2085854536 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70011d20 con 0x7f3d8015b370 2023-11-24T06:26:05.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 -- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d8015b370 msgr2=0x7f3d8015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d8015b370 0x7f3d8015b800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d70009b80 comp rx=0 tx=0).stop 2023-11-24T06:26:05.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 -- 172.21.15.114:0/2085854536 shutdown_connections 2023-11-24T06:26:05.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d800b3c60 0x7f3d800b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/2085854536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d8015b370 0x7f3d8015b800 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 -- 172.21.15.114:0/2085854536 >> 172.21.15.114:0/2085854536 conn(0x7f3d800d0bf0 msgr2=0x7f3d800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 -- 172.21.15.114:0/2085854536 shutdown_connections 2023-11-24T06:26:05.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.522+0000 7f3d87bf9700 1 -- 172.21.15.114:0/2085854536 wait complete. 2023-11-24T06:26:05.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.523+0000 7f3d87bf9700 1 Processor -- start 2023-11-24T06:26:05.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 -- start start 2023-11-24T06:26:05.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d800b3c60 0x7f3d8014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d8015b370 0x7f3d80074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8009b0c0 con 0x7f3d801587f0 2023-11-24T06:26:05.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80159b30 con 0x7f3d800b3c60 2023-11-24T06:26:05.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d87bf9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80159e10 con 0x7f3d8015b370 2023-11-24T06:26:05.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d85194700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d85194700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41162/0 (socket says 172.21.15.114:41162) 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d85194700 1 -- 172.21.15.114:0/3134114028 learned_addr learned my addr 172.21.15.114:0/3134114028 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d85995700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d800b3c60 0x7f3d8014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.524+0000 7f3d86196700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d8015b370 0x7f3d80074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 -- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d8015b370 msgr2=0x7f3d80074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d8015b370 0x7f3d80074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 -- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d800b3c60 msgr2=0x7f3d8014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d800b3c60 0x7f3d8014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 -- 172.21.15.114:0/3134114028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d700094a0 con 0x7f3d801587f0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d85194700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700036a0 con 0x7f3d801587f0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7001e070 con 0x7f3d801587f0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8009aa00 con 0x7f3d801587f0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70003a20 con 0x7f3d801587f0 2023-11-24T06:26:05.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.525+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8014cb70 con 0x7f3d801587f0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.526+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8009b330 con 0x7f3d801587f0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.526+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3d70003b80 con 0x7f3d801587f0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.527+0000 7f3d767fc700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 0x7f3d6406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.527+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3d7008a7b0 con 0x7f3d801587f0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.527+0000 7f3d85995700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 0x7f3d6406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.527+0000 7f3d85995700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 0x7f3d6406d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007fd0 tx=0x7f3d7c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.532+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d70055630 con 0x7f3d801587f0 2023-11-24T06:26:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.723+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d80068fb0 con 0x7f3d801587f0 2023-11-24T06:26:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.724+0000 7f3d767fc700 1 -- 172.21.15.114:0/3134114028 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d70015020 con 0x7f3d801587f0 2023-11-24T06:26:05.728 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:26:05.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 msgr2=0x7f3d6406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 0x7f3d6406d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007fd0 tx=0x7f3d7c007de0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 msgr2=0x7f3d80074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000c00 tx=0x7f3d70004750 comp rx=0 tx=0).stop 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 shutdown_connections 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3d6406b080 0x7f3d6406d540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d8015b370 0x7f3d80074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d800b3c60 0x7f3d8014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 --2- 172.21.15.114:0/3134114028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d801587f0 0x7f3d80074530 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 >> 172.21.15.114:0/3134114028 conn(0x7f3d800d0bf0 msgr2=0x7f3d800b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 shutdown_connections 2023-11-24T06:26:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.726+0000 7f3d87bf9700 1 -- 172.21.15.114:0/3134114028 wait complete. 2023-11-24T06:26:05.741 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:26:05.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.861+0000 7f220e29a700 1 Processor -- start 2023-11-24T06:26:05.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.861+0000 7f220e29a700 1 -- start start 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220e29a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22080746a0 0x7f22080b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220e29a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22080afc90 0x7f220806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220e29a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22080b3c60 con 0x7f220815e160 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220e29a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080b4500 con 0x7f22080afc90 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220e29a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080b58c0 con 0x7f22080746a0 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22080afc90 0x7f220806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22080afc90 0x7f220806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41172/0 (socket says 172.21.15.114:41172) 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 -- 172.21.15.114:0/1215339903 learned_addr learned my addr 172.21.15.114:0/1215339903 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f22077fe700 1 --2- 172.21.15.114:0/1215339903 >> v2:172.21.15.172:3300/0 conn(0x7f22080746a0 0x7f22080b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 -- 172.21.15.114:0/1215339903 >> v2:172.21.15.172:3300/0 conn(0x7f22080746a0 msgr2=0x7f22080b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 --2- 172.21.15.114:0/1215339903 >> v2:172.21.15.172:3300/0 conn(0x7f22080746a0 0x7f22080b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 -- 172.21.15.114:0/1215339903 >> v1:172.21.15.172:6789/0 conn(0x7f220815e160 legacy=0x7f2208073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 -- 172.21.15.114:0/1215339903 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220806bdc0 con 0x7f22080afc90 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.862+0000 7f220c837700 1 --2- 172.21.15.114:0/1215339903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080afc90 0x7f220806d930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f22080d0880 tx=0x7f220000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbef2af0b61f9139 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f2206ffd700 1 -- 172.21.15.114:0/1215339903 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220000e070 con 0x7f22080afc90 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f2206ffd700 1 -- 172.21.15.114:0/1215339903 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200003a20 con 0x7f22080afc90 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f2206ffd700 1 -- 172.21.15.114:0/1215339903 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200003dd0 con 0x7f22080afc90 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 -- 172.21.15.114:0/1215339903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080afc90 msgr2=0x7f220806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 --2- 172.21.15.114:0/1215339903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080afc90 0x7f220806d930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f22080d0880 tx=0x7f220000b330 comp rx=0 tx=0).stop 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 -- 172.21.15.114:0/1215339903 shutdown_connections 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 --2- 172.21.15.114:0/1215339903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22080afc90 0x7f220806d930 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 --2- 172.21.15.114:0/1215339903 >> v2:172.21.15.172:3300/0 conn(0x7f22080746a0 0x7f22080b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 -- 172.21.15.114:0/1215339903 >> 172.21.15.114:0/1215339903 conn(0x7f22080d0bf0 msgr2=0x7f22080d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:05.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 -- 172.21.15.114:0/1215339903 shutdown_connections 2023-11-24T06:26:05.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.863+0000 7f220e29a700 1 -- 172.21.15.114:0/1215339903 wait complete. 2023-11-24T06:26:05.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.864+0000 7f220e29a700 1 Processor -- start 2023-11-24T06:26:05.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 -- start start 2023-11-24T06:26:05.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 0x7f220815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 0x7f2208160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.871 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220815fde0 con 0x7f220815e160 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220815fbb0 con 0x7f22080afc90 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220e29a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2208157a90 con 0x7f22080746a0 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.865+0000 7f220c837700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41178/0 (socket says 172.21.15.114:41178) 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 -- 172.21.15.114:0/948071699 learned_addr learned my addr 172.21.15.114:0/948071699 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:05.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f22077fe700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 0x7f2208160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f2207fff700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 0x7f220815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 -- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 msgr2=0x7f220815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 0x7f220815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 -- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 msgr2=0x7f2208160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 0x7f2208160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 -- 172.21.15.114:0/948071699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009b30 con 0x7f220815e160 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f22077fe700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 0x7f2208160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f2207fff700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 0x7f220815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220c837700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f220000b370 tx=0x7f2200004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004440 con 0x7f220815e160 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22000045a0 con 0x7f220815e160 2023-11-24T06:26:05.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2208157d70 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004950 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.866+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2208158330 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.867+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22081584c0 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.867+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2200012430 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.868+0000 7f2204ff9700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 0x7f21f406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.868+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f220008ae20 con 0x7f220815e160 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.868+0000 7f2207fff700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 0x7f21f406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:05.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.868+0000 7f2207fff700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 0x7f21f406d540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:05.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:05.872+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2200055b90 con 0x7f220815e160 2023-11-24T06:26:06.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.060+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2208068fb0 con 0x7f220815e160 2023-11-24T06:26:06.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.060+0000 7f2204ff9700 1 -- 172.21.15.114:0/948071699 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22000170a0 con 0x7f220815e160 2023-11-24T06:26:06.064 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 msgr2=0x7f21f406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 0x7f21f406d540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc008040 comp rx=0 tx=0).stop 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 msgr2=0x7f220815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f220000b370 tx=0x7f2200004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 shutdown_connections 2023-11-24T06:26:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f21f406b080 0x7f21f406d540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f220815e160 0x7f220815af60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22080746a0 0x7f220815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.062+0000 7f220e29a700 1 --2- 172.21.15.114:0/948071699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22080afc90 0x7f2208160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.063+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 >> 172.21.15.114:0/948071699 conn(0x7f22080d0bf0 msgr2=0x7f22080b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.063+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 shutdown_connections 2023-11-24T06:26:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.063+0000 7f220e29a700 1 -- 172.21.15.114:0/948071699 wait complete. 2023-11-24T06:26:06.078 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:26:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.195+0000 7f4f5e90a700 1 Processor -- start 2023-11-24T06:26:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.195+0000 7f4f5e90a700 1 -- start start 2023-11-24T06:26:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f5e90a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f5815e080 0x7f4f580b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f5e90a700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580ae430 con 0x7f4f580af4d0 2023-11-24T06:26:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f5e90a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580ae710 con 0x7f4f5806c710 2023-11-24T06:26:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f5e90a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580af0b0 con 0x7f4f5815e080 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f57fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f5815e080 0x7f4f580b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f577fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4f580af4d0 0x7f4f5806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56892/0 (socket says 172.21.15.114:56892) 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f577fe700 1 -- 172.21.15.114:0/462210242 learned_addr learned my addr 172.21.15.114:0/462210242 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f57fff700 1 -- 172.21.15.114:0/462210242 >> v1:172.21.15.172:6789/0 conn(0x7f4f5806c710 legacy=0x7f4f5806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f57fff700 1 -- 172.21.15.114:0/462210242 >> v1:172.21.15.114:6789/0 conn(0x7f4f580af4d0 legacy=0x7f4f5806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f56ffd700 1 -- 172.21.15.114:0/462210242 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647306411 0 0) 0x7f4f580ae430 con 0x7f4f580af4d0 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f57fff700 1 -- 172.21.15.114:0/462210242 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580ae9f0 con 0x7f4f5815e080 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f57fff700 1 --2- 172.21.15.114:0/462210242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5815e080 0x7f4f580b5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009840 tx=0x7f4f48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d888771e6caf75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f56ffd700 1 -- 172.21.15.114:0/462210242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004230 con 0x7f4f5815e080 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f56ffd700 1 -- 172.21.15.114:0/462210242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48004390 con 0x7f4f5815e080 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.196+0000 7f4f56ffd700 1 -- 172.21.15.114:0/462210242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f480036a0 con 0x7f4f5815e080 2023-11-24T06:26:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 -- 172.21.15.114:0/462210242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5815e080 msgr2=0x7f4f580b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/462210242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5815e080 0x7f4f580b5050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009840 tx=0x7f4f48009b50 comp rx=0 tx=0).stop 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 -- 172.21.15.114:0/462210242 shutdown_connections 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/462210242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5815e080 0x7f4f580b5050 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 -- 172.21.15.114:0/462210242 >> 172.21.15.114:0/462210242 conn(0x7f4f58071b90 msgr2=0x7f4f58071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 -- 172.21.15.114:0/462210242 shutdown_connections 2023-11-24T06:26:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.197+0000 7f4f5e90a700 1 -- 172.21.15.114:0/462210242 wait complete. 2023-11-24T06:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.198+0000 7f4f5e90a700 1 Processor -- start 2023-11-24T06:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.198+0000 7f4f5e90a700 1 -- start start 2023-11-24T06:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 0x7f4f581570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 0x7f4f581595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58161330 con 0x7f4f5806c710 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58160460 con 0x7f4f5815e080 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5e90a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581606b0 con 0x7f4f580af4d0 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41188/0 (socket says 172.21.15.114:41188) 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 -- 172.21.15.114:0/3353163258 learned_addr learned my addr 172.21.15.114:0/3353163258 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5cea7700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 0x7f4f581595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f577fe700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 0x7f4f581570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 -- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 msgr2=0x7f4f581570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 0x7f4f581570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 -- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 msgr2=0x7f4f581595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 0x7f4f581595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f57fff700 1 -- 172.21.15.114:0/3353163258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480094a0 con 0x7f4f5806c710 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f5cea7700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 0x7f4f581595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.199+0000 7f4f577fe700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 0x7f4f581570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f57fff700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4f480090b0 tx=0x7f4f480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48011830 con 0x7f4f5806c710 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48011990 con 0x7f4f5806c710 2023-11-24T06:26:06.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f58160990 con 0x7f4f5806c710 2023-11-24T06:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48011d40 con 0x7f4f5806c710 2023-11-24T06:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.200+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f58160f50 con 0x7f4f5806c710 2023-11-24T06:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.201+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f580753f0 con 0x7f4f5806c710 2023-11-24T06:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.201+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4f4801b440 con 0x7f4f5806c710 2023-11-24T06:26:06.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.201+0000 7f4f54ff9700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 0x7f4f4406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.201+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f4808b290 con 0x7f4f5806c710 2023-11-24T06:26:06.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.201+0000 7f4f577fe700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 0x7f4f4406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.202+0000 7f4f577fe700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 0x7f4f4406d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f58156cd0 tx=0x7f4f4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.206+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f48056110 con 0x7f4f5806c710 2023-11-24T06:26:06.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.412+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f58067f60 con 0x7f4f5806c710 2023-11-24T06:26:06.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.412+0000 7f4f54ff9700 1 -- 172.21.15.114:0/3353163258 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4f48059730 con 0x7f4f5806c710 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 msgr2=0x7f4f4406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 0x7f4f4406d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f58156cd0 tx=0x7f4f4c009040 comp rx=0 tx=0).stop 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 msgr2=0x7f4f581f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4f480090b0 tx=0x7f4f480047c0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 shutdown_connections 2023-11-24T06:26:06.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f4406b080 0x7f4f4406d540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f5815e080 0x7f4f581595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f5806c710 0x7f4f581f6540 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 --2- 172.21.15.114:0/3353163258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f580af4d0 0x7f4f581570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 >> 172.21.15.114:0/3353163258 conn(0x7f4f58071b90 msgr2=0x7f4f580b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 shutdown_connections 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.414+0000 7f4f5e90a700 1 -- 172.21.15.114:0/3353163258 wait complete. 2023-11-24T06:26:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:26:06.430 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:26:06.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:06.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:26:06.473 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:26:06.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:06.473 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:26:06.530 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:26:06.531 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:06.553 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:26:06.553 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:26:06.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.731+0000 7fd4b80ca700 1 Processor -- start 2023-11-24T06:26:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.731+0000 7fd4b80ca700 1 -- start start 2023-11-24T06:26:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b015bd60 0x7fd4b00d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4b015c140 0x7fd4b00cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 0x7fd4b00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00cc780 con 0x7fd4b015c140 2023-11-24T06:26:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00cc9b0 con 0x7fd4b00cf2b0 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b80ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00ccc90 con 0x7fd4b015bd60 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4b015c140 0x7fd4b00cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4b015c140 0x7fd4b00cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41196/0 (socket says 172.21.15.114:41196) 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 -- 172.21.15.114:0/685263626 learned_addr learned my addr 172.21.15.114:0/685263626 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b6667700 1 --2- 172.21.15.114:0/685263626 >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 0x7fd4b00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5e66700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b015bd60 0x7fd4b00d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 -- 172.21.15.114:0/685263626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b015bd60 msgr2=0x7fd4b00d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b015bd60 0x7fd4b00d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 -- 172.21.15.114:0/685263626 >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 msgr2=0x7fd4b00c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 --2- 172.21.15.114:0/685263626 >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 0x7fd4b00c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 -- 172.21.15.114:0/685263626 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00ccf70 con 0x7fd4b015c140 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b6667700 1 --2- 172.21.15.114:0/685263626 >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 0x7fd4b00c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b5665700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015c140 0x7fd4b00cda50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006ebc0 tx=0x7fd4ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4ad664a359c306 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b4e64700 1 -- 172.21.15.114:0/685263626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00b870 con 0x7fd4b015c140 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b4e64700 1 -- 172.21.15.114:0/685263626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00b9d0 con 0x7fd4b015c140 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.732+0000 7fd4b4e64700 1 -- 172.21.15.114:0/685263626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00bd60 con 0x7fd4b015c140 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 -- 172.21.15.114:0/685263626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015c140 msgr2=0x7fd4b00cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015c140 0x7fd4b00cda50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006ebc0 tx=0x7fd4ac009bd0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 -- 172.21.15.114:0/685263626 shutdown_connections 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b015bd60 0x7fd4b00d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/685263626 >> v2:172.21.15.172:3300/0 conn(0x7fd4b00cf2b0 0x7fd4b00c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/685263626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015c140 0x7fd4b00cda50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 -- 172.21.15.114:0/685263626 >> 172.21.15.114:0/685263626 conn(0x7fd4b0073130 msgr2=0x7fd4b009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 -- 172.21.15.114:0/685263626 shutdown_connections 2023-11-24T06:26:06.740 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.733+0000 7fd4b80ca700 1 -- 172.21.15.114:0/685263626 wait complete. 2023-11-24T06:26:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.734+0000 7fd4b80ca700 1 Processor -- start 2023-11-24T06:26:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 -- start start 2023-11-24T06:26:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 0x7fd4b01561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 0x7fd4b00b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0156700 con 0x7fd4b015bd60 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0158f70 con 0x7fd4b015c140 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.735+0000 7fd4b80ca700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0159250 con 0x7fd4b00cf2b0 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41204/0 (socket says 172.21.15.114:41204) 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 -- 172.21.15.114:0/3008745088 learned_addr learned my addr 172.21.15.114:0/3008745088 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5e66700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 0x7fd4b01561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b6667700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 0x7fd4b00b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 -- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 msgr2=0x7fd4b01561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 0x7fd4b01561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 -- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 msgr2=0x7fd4b00b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 0x7fd4b00b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 -- 172.21.15.114:0/3008745088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac009550 con 0x7fd4b015bd60 2023-11-24T06:26:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5e66700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 0x7fd4b01561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b6667700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 0x7fd4b00b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b5665700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac000c00 tx=0x7fd4ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac004520 con 0x7fd4b015bd60 2023-11-24T06:26:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac01e070 con 0x7fd4b015bd60 2023-11-24T06:26:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b0159530 con 0x7fd4b015bd60 2023-11-24T06:26:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0038c0 con 0x7fd4b015bd60 2023-11-24T06:26:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.736+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b0159af0 con 0x7fd4b015bd60 2023-11-24T06:26:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.737+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b0067f60 con 0x7fd4b015bd60 2023-11-24T06:26:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.738+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd4ac003a20 con 0x7fd4b015bd60 2023-11-24T06:26:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.738+0000 7fd4a67fc700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 0x7fd49406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.738+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd4ac020070 con 0x7fd4b015bd60 2023-11-24T06:26:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.738+0000 7fd4b5e66700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 0x7fd49406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.738+0000 7fd4b5e66700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 0x7fd49406d540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009970 tx=0x7fd4a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.742+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4ac055bf0 con 0x7fd4b015bd60 2023-11-24T06:26:06.939 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.935+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b00b7bf0 con 0x7fd4b015bd60 2023-11-24T06:26:06.940 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.936+0000 7fd4a67fc700 1 -- 172.21.15.114:0/3008745088 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fd4ac015090 con 0x7fd4b015bd60 2023-11-24T06:26:06.940 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:06.940 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:59.863101+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 msgr2=0x7fd49406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 0x7fd49406d540 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009970 tx=0x7fd4a0008040 comp rx=0 tx=0).stop 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 msgr2=0x7fd4b00b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac000c00 tx=0x7fd4ac004750 comp rx=0 tx=0).stop 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 shutdown_connections 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd49406b080 0x7fd49406d540 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.942 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4b015c140 0x7fd4b00b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4b00cf2b0 0x7fd4b01561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 --2- 172.21.15.114:0/3008745088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4b015bd60 0x7fd4b00b6d10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:06.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.938+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 >> 172.21.15.114:0/3008745088 conn(0x7fd4b0073130 msgr2=0x7fd4b00d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:06.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.939+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 shutdown_connections 2023-11-24T06:26:06.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:06.939+0000 7fd4b80ca700 1 -- 172.21.15.114:0/3008745088 wait complete. 2023-11-24T06:26:06.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:48.353169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:48.873132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:26:06.955 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:26:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.074+0000 7feb04ebf700 1 Processor -- start 2023-11-24T06:26:07.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.074+0000 7feb04ebf700 1 -- start start 2023-11-24T06:26:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb0015bf50 0x7feb000ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb000ce830 0x7feb000cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000cf1f0 0x7feb000c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0015c330 con 0x7feb0015bf50 2023-11-24T06:26:07.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000d0140 con 0x7feb000ce830 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feb04ebf700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb000d03e0 con 0x7feb000cf1f0 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feafe59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb0015bf50 0x7feb000ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feafe59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feb0015bf50 0x7feb000ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41210/0 (socket says 172.21.15.114:41210) 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feafe59c700 1 -- 172.21.15.114:0/1685956571 learned_addr learned my addr 172.21.15.114:0/1685956571 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.075+0000 7feafdd9b700 1 --2- 172.21.15.114:0/1685956571 >> v2:172.21.15.172:3300/0 conn(0x7feb000ce830 0x7feb000cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafed9d700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000cf1f0 0x7feb000c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 -- 172.21.15.114:0/1685956571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000cf1f0 msgr2=0x7feb000c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000cf1f0 0x7feb000c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 -- 172.21.15.114:0/1685956571 >> v2:172.21.15.172:3300/0 conn(0x7feb000ce830 msgr2=0x7feb000cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 --2- 172.21.15.114:0/1685956571 >> v2:172.21.15.172:3300/0 conn(0x7feb000ce830 0x7feb000cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 -- 172.21.15.114:0/1685956571 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb000d06c0 con 0x7feb0015bf50 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafe59c700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb0015bf50 0x7feb000ce2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7feb00072420 tx=0x7feaf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79246052a207bb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafd59a700 1 -- 172.21.15.114:0/1685956571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000b870 con 0x7feb0015bf50 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafd59a700 1 -- 172.21.15.114:0/1685956571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000b9d0 con 0x7feb0015bf50 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feafd59a700 1 -- 172.21.15.114:0/1685956571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000bd60 con 0x7feb0015bf50 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feb04ebf700 1 -- 172.21.15.114:0/1685956571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb0015bf50 msgr2=0x7feb000ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.076+0000 7feb04ebf700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb0015bf50 0x7feb000ce2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7feb00072420 tx=0x7feaf0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 -- 172.21.15.114:0/1685956571 shutdown_connections 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000cf1f0 0x7feb000c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 --2- 172.21.15.114:0/1685956571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb0015bf50 0x7feb000ce2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 --2- 172.21.15.114:0/1685956571 >> v2:172.21.15.172:3300/0 conn(0x7feb000ce830 0x7feb000cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 -- 172.21.15.114:0/1685956571 >> 172.21.15.114:0/1685956571 conn(0x7feb000b3780 msgr2=0x7feb000b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 -- 172.21.15.114:0/1685956571 shutdown_connections 2023-11-24T06:26:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.077+0000 7feb04ebf700 1 -- 172.21.15.114:0/1685956571 wait complete. 2023-11-24T06:26:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.078+0000 7feb04ebf700 1 Processor -- start 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.078+0000 7feb04ebf700 1 -- start start 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000ce830 0x7feb00155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb0015bf50 0x7feb00159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000b6600 con 0x7feb000cf1f0 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00156b80 con 0x7feb0015bf50 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb00156e60 con 0x7feb000ce830 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41218/0 (socket says 172.21.15.114:41218) 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 -- 172.21.15.114:0/3705761931 learned_addr learned my addr 172.21.15.114:0/3705761931 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafe59c700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000ce830 0x7feb00155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafed9d700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb0015bf50 0x7feb00159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 -- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000ce830 msgr2=0x7feb00155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000ce830 0x7feb00155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 -- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb0015bf50 msgr2=0x7feb00159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb0015bf50 0x7feb00159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 -- 172.21.15.114:0/3705761931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009550 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feafdd9b700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e810 tx=0x7feaf400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400c070 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4013610 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb00157140 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40139c0 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.079+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb00148710 con 0x7feb000cf1f0 2023-11-24T06:26:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.080+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb00148900 con 0x7feb000cf1f0 2023-11-24T06:26:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.081+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7feaf4013b20 con 0x7feb000cf1f0 2023-11-24T06:26:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.081+0000 7feaeeffd700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 0x7feae006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.081+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feaf4089800 con 0x7feb000cf1f0 2023-11-24T06:26:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.081+0000 7feafe59c700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 0x7feae006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.082+0000 7feafe59c700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 0x7feae006d540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feaf0014040 tx=0x7feaf0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.086+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feaf4054450 con 0x7feb000cf1f0 2023-11-24T06:26:07.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.275+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb00068fb0 con 0x7feb000cf1f0 2023-11-24T06:26:07.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.275+0000 7feaeeffd700 1 -- 172.21.15.114:0/3705761931 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7feaf4057a70 con 0x7feb000cf1f0 2023-11-24T06:26:07.280 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:26:07.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.277+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 msgr2=0x7feae006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.281 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.277+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 0x7feae006d540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feaf0014040 tx=0x7feaf0008030 comp rx=0 tx=0).stop 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.277+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 msgr2=0x7feb001f1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.277+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e810 tx=0x7feaf400eb20 comp rx=0 tx=0).stop 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 shutdown_connections 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feae006b080 0x7feae006d540 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb0015bf50 0x7feb00159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb000ce830 0x7feb00155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 --2- 172.21.15.114:0/3705761931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb000cf1f0 0x7feb001f1670 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 >> 172.21.15.114:0/3705761931 conn(0x7feb000b3780 msgr2=0x7feb000702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 shutdown_connections 2023-11-24T06:26:07.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.278+0000 7feb04ebf700 1 -- 172.21.15.114:0/3705761931 wait complete. 2023-11-24T06:26:07.293 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:26:07.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.413+0000 7f240629e700 1 Processor -- start 2023-11-24T06:26:07.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.413+0000 7f240629e700 1 -- start start 2023-11-24T06:26:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f240629e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 0x7f24000af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f240629e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24000b4e00 0x7f24000b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f240629e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24000b4970 con 0x7f24000cd840 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f240629e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24000cfed0 con 0x7f24000b4e00 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f240629e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24000b4360 con 0x7f240015be20 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24000b4e00 0x7f24000b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24000b4e00 0x7f24000b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41226/0 (socket says 172.21.15.114:41226) 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 -- 172.21.15.114:0/3973620279 learned_addr learned my addr 172.21.15.114:0/3973620279 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23fffff700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 0x7f24000af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 -- 172.21.15.114:0/3973620279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 msgr2=0x7f24000af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 0x7f24000af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 -- 172.21.15.114:0/3973620279 >> v1:172.21.15.172:6789/0 conn(0x7f24000cd840 legacy=0x7f240015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 -- 172.21.15.114:0/3973620279 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240006f940 con 0x7f24000b4e00 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23fffff700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 0x7f24000af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.414+0000 7f23ff7fe700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24000b4e00 0x7f24000b5280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f23f00098f0 tx=0x7f23f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520aeb3e4d8a5652 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f23feffd700 1 -- 172.21.15.114:0/3973620279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0004230 con 0x7f24000b4e00 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f23feffd700 1 -- 172.21.15.114:0/3973620279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f0004390 con 0x7f24000b4e00 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f23feffd700 1 -- 172.21.15.114:0/3973620279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00036a0 con 0x7f24000b4e00 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 -- 172.21.15.114:0/3973620279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24000b4e00 msgr2=0x7f24000b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24000b4e00 0x7f24000b5280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f23f00098f0 tx=0x7f23f0009c00 comp rx=0 tx=0).stop 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 -- 172.21.15.114:0/3973620279 shutdown_connections 2023-11-24T06:26:07.425 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f240015be20 0x7f24000af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 --2- 172.21.15.114:0/3973620279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24000b4e00 0x7f24000b5280 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 -- 172.21.15.114:0/3973620279 >> 172.21.15.114:0/3973620279 conn(0x7f24000b2050 msgr2=0x7f24000b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 -- 172.21.15.114:0/3973620279 shutdown_connections 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.415+0000 7f240629e700 1 -- 172.21.15.114:0/3973620279 wait complete. 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.416+0000 7f240629e700 1 Processor -- start 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 -- start start 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 0x7f2400150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 0x7f24000ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.426 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240015c960 con 0x7f240015be20 2023-11-24T06:26:07.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2400154e20 con 0x7f24000b4e00 2023-11-24T06:26:07.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240629e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24001514f0 con 0x7f24000cd840 2023-11-24T06:26:07.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240483b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.427 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240483b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41232/0 (socket says 172.21.15.114:41232) 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.417+0000 7f240483b700 1 -- 172.21.15.114:0/2468898070 learned_addr learned my addr 172.21.15.114:0/2468898070 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23fffff700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 0x7f2400150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23ff7fe700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 0x7f24000ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 -- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 msgr2=0x7f24000ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 0x7f24000ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 -- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 msgr2=0x7f2400150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 0x7f2400150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 -- 172.21.15.114:0/2468898070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f0009550 con 0x7f240015be20 2023-11-24T06:26:07.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23fffff700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 0x7f2400150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23ff7fe700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 0x7f24000ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240483b700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b970 tx=0x7f23f800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800c9b0 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f800cb10 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2400152530 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8014440 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.418+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240015c2a0 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.419+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2400068d90 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.419+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f23f801d3b0 con 0x7f240015be20 2023-11-24T06:26:07.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.420+0000 7f23fcff9700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 0x7f23ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.420+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23f80895c0 con 0x7f240015be20 2023-11-24T06:26:07.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.420+0000 7f23fffff700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 0x7f23ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.420+0000 7f23fffff700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 0x7f23ec06d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23f40089b0 tx=0x7f23f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.424+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23f80543b0 con 0x7f240015be20 2023-11-24T06:26:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.612+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f240015caa0 con 0x7f240015be20 2023-11-24T06:26:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.613+0000 7f23fcff9700 1 -- 172.21.15.114:0/2468898070 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f240015caa0 con 0x7f240015be20 2023-11-24T06:26:07.616 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:26:07.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 msgr2=0x7f23ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 0x7f23ec06d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23f40089b0 tx=0x7f23f4008040 comp rx=0 tx=0).stop 2023-11-24T06:26:07.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 msgr2=0x7f24000aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b970 tx=0x7f23f800bd30 comp rx=0 tx=0).stop 2023-11-24T06:26:07.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 shutdown_connections 2023-11-24T06:26:07.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f23ec06b080 0x7f23ec06d540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f240015be20 0x7f24000aee00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24000b4e00 0x7f2400150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 --2- 172.21.15.114:0/2468898070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24000cd840 0x7f24000ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 >> 172.21.15.114:0/2468898070 conn(0x7f24000b2050 msgr2=0x7f24000afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 shutdown_connections 2023-11-24T06:26:07.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.615+0000 7f240629e700 1 -- 172.21.15.114:0/2468898070 wait complete. 2023-11-24T06:26:07.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:26:07.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.750+0000 7f0fc5c6b700 1 Processor -- start 2023-11-24T06:26:07.754 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.750+0000 7f0fc5c6b700 1 -- start start 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fc5c6b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0fc00afb40 0x7f0fc00cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fc5c6b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc00cccd0 con 0x7f0fc00cfa80 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fc5c6b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc0062440 con 0x7f0fc015ada0 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fc5c6b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00cd2e0 con 0x7f0fc00afb40 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0fc00afb40 0x7f0fc00cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0fc00afb40 0x7f0fc00cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41238/0 (socket says 172.21.15.114:41238) 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fbffff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0fc00cfa80 0x7f0fc006d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56946/0 (socket says 172.21.15.114:56946) 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 -- 172.21.15.114:0/3134297173 learned_addr learned my addr 172.21.15.114:0/3134297173 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 -- 172.21.15.114:0/3134297173 >> v1:172.21.15.172:6789/0 conn(0x7f0fc015ada0 legacy=0x7f0fc006f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 -- 172.21.15.114:0/3134297173 >> v1:172.21.15.114:6789/0 conn(0x7f0fc00cfa80 legacy=0x7f0fc006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fb6ffd700 1 -- 172.21.15.114:0/3134297173 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0062720 con 0x7f0fc00afb40 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.751+0000 7f0fbeffd700 1 -- 172.21.15.114:0/3134297173 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063222375 0 0) 0x7f0fc00cccd0 con 0x7f0fc00cfa80 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fb6ffd700 1 --2- 172.21.15.114:0/3134297173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00afb40 0x7f0fc00cf540 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800bb90 tx=0x7f0fa800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675d2d5f8f28ae60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fbeffd700 1 -- 172.21.15.114:0/3134297173 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800dd10 con 0x7f0fc00afb40 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fbeffd700 1 -- 172.21.15.114:0/3134297173 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa800de70 con 0x7f0fc00afb40 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fbeffd700 1 -- 172.21.15.114:0/3134297173 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004230 con 0x7f0fc00afb40 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/3134297173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00afb40 msgr2=0x7f0fc00cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/3134297173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00afb40 0x7f0fc00cf540 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800bb90 tx=0x7f0fa800bea0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/3134297173 shutdown_connections 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/3134297173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc00afb40 0x7f0fc00cf540 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/3134297173 >> 172.21.15.114:0/3134297173 conn(0x7f0fc00b2050 msgr2=0x7f0fc00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/3134297173 shutdown_connections 2023-11-24T06:26:07.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.752+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/3134297173 wait complete. 2023-11-24T06:26:07.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.754+0000 7f0fc5c6b700 1 Processor -- start 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.754+0000 7f0fc5c6b700 1 -- start start 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc00afb40 0x7f0fc014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc00cfa80 0x7f0fc015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0154e10 con 0x7f0fc015ada0 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc015cd80 con 0x7f0fc00afb40 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fc5c6b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc0155060 con 0x7f0fc00cfa80 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41246/0 (socket says 172.21.15.114:41246) 2023-11-24T06:26:07.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 -- 172.21.15.114:0/2636126841 learned_addr learned my addr 172.21.15.114:0/2636126841 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:07.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fb6ffd700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc00cfa80 0x7f0fc015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbf7fe700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc00afb40 0x7f0fc014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 -- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc00cfa80 msgr2=0x7f0fc015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc00cfa80 0x7f0fc015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 -- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc00afb40 msgr2=0x7f0fc014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc00afb40 0x7f0fc014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 -- 172.21.15.114:0/2636126841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa800b7f0 con 0x7f0fc015ada0 2023-11-24T06:26:07.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.755+0000 7f0fbffff700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800cca0 tx=0x7f0fb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.756+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb800cea0 con 0x7f0fc015ada0 2023-11-24T06:26:07.762 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.756+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb800ade0 con 0x7f0fc015ada0 2023-11-24T06:26:07.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.756+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8007310 con 0x7f0fc015ada0 2023-11-24T06:26:07.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.756+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc0155340 con 0x7f0fc015ada0 2023-11-24T06:26:07.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.756+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc0155900 con 0x7f0fc015ada0 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.757+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc01f7ed0 con 0x7f0fc015ada0 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.757+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0fb8007470 con 0x7f0fc015ada0 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.758+0000 7f0fbcff9700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 0x7f0fa006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.758+0000 7f0fbf7fe700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 0x7f0fa006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.758+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fb8089580 con 0x7f0fc015ada0 2023-11-24T06:26:07.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.758+0000 7f0fbf7fe700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 0x7f0fa006d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00b1c40 tx=0x7f0fb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:07.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.762+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fb8054400 con 0x7f0fc015ada0 2023-11-24T06:26:07.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.954+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fc0068d90 con 0x7f0fc015ada0 2023-11-24T06:26:07.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.954+0000 7f0fbcff9700 1 -- 172.21.15.114:0/2636126841 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fb8018020 con 0x7f0fc015ada0 2023-11-24T06:26:07.958 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 msgr2=0x7f0fa006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 0x7f0fa006d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00b1c40 tx=0x7f0fb0008040 comp rx=0 tx=0).stop 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 msgr2=0x7f0fc0159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800cca0 tx=0x7f0fb800a340 comp rx=0 tx=0).stop 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 shutdown_connections 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fa006b080 0x7f0fa006d540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fc015ada0 0x7f0fc0159ce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fc00afb40 0x7f0fc014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 --2- 172.21.15.114:0/2636126841 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0fc00cfa80 0x7f0fc015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:07.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.956+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 >> 172.21.15.114:0/2636126841 conn(0x7f0fc00b2050 msgr2=0x7f0fc006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:07.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.957+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 shutdown_connections 2023-11-24T06:26:07.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:07.957+0000 7f0fc5c6b700 1 -- 172.21.15.114:0/2636126841 wait complete. 2023-11-24T06:26:07.972 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:26:08.094 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.090+0000 7f2b6e639700 1 Processor -- start 2023-11-24T06:26:08.094 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.090+0000 7f2b6e639700 1 -- start start 2023-11-24T06:26:08.097 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6e639700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 0x7f2b680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6e639700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2b6806c710 0x7f2b680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6e639700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680ae430 con 0x7f2b680af4d0 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6e639700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680af0b0 con 0x7f2b6806c710 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6e639700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680b05a0 con 0x7f2b6815e080 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6cbd6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2b6806c710 0x7f2b680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6cbd6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2b6806c710 0x7f2b680afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41254/0 (socket says 172.21.15.114:41254) 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.091+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/3014060381 learned_addr learned my addr 172.21.15.114:0/3014060381 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b67fff700 1 --2- 172.21.15.114:0/3014060381 >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 0x7f2b680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/3014060381 >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 msgr2=0x7f2b680b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:08.098 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b6cbd6700 1 --2- 172.21.15.114:0/3014060381 >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 0x7f2b680b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/3014060381 >> v1:172.21.15.172:6789/0 conn(0x7f2b680af4d0 legacy=0x7f2b6806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/3014060381 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680ae710 con 0x7f2b6806c710 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b67fff700 1 --2- 172.21.15.114:0/3014060381 >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 0x7f2b680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b6cbd6700 1 --2- 172.21.15.114:0/3014060381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6806c710 0x7f2b680afb90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2b68072a90 tx=0x7f2b6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98e675ae2102c6b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b66ffd700 1 -- 172.21.15.114:0/3014060381 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000e070 con 0x7f2b6806c710 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b66ffd700 1 -- 172.21.15.114:0/3014060381 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000b980 con 0x7f2b6806c710 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.092+0000 7f2b66ffd700 1 -- 172.21.15.114:0/3014060381 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000bd30 con 0x7f2b6806c710 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 -- 172.21.15.114:0/3014060381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6806c710 msgr2=0x7f2b680afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 --2- 172.21.15.114:0/3014060381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6806c710 0x7f2b680afb90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2b68072a90 tx=0x7f2b6000b330 comp rx=0 tx=0).stop 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 -- 172.21.15.114:0/3014060381 shutdown_connections 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 --2- 172.21.15.114:0/3014060381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6806c710 0x7f2b680afb90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.099 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 --2- 172.21.15.114:0/3014060381 >> v2:172.21.15.172:3300/0 conn(0x7f2b6815e080 0x7f2b680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 -- 172.21.15.114:0/3014060381 >> 172.21.15.114:0/3014060381 conn(0x7f2b68071b90 msgr2=0x7f2b68071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 -- 172.21.15.114:0/3014060381 shutdown_connections 2023-11-24T06:26:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.093+0000 7f2b6e639700 1 -- 172.21.15.114:0/3014060381 wait complete. 2023-11-24T06:26:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.094+0000 7f2b6e639700 1 Processor -- start 2023-11-24T06:26:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.094+0000 7f2b6e639700 1 -- start start 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b6806c710 0x7f2b68153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 0x7f2b68157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68153bd0 con 0x7f2b6815e080 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68153f20 con 0x7f2b6806c710 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68154200 con 0x7f2b680af4d0 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41260/0 (socket says 172.21.15.114:41260) 2023-11-24T06:26:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/1551159864 learned_addr learned my addr 172.21.15.114:0/1551159864 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b67fff700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b6806c710 0x7f2b68153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b677fe700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 0x7f2b68157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 msgr2=0x7f2b68157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 0x7f2b68157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b6806c710 msgr2=0x7f2b68153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b6806c710 0x7f2b68153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 -- 172.21.15.114:0/1551159864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009b30 con 0x7f2b6815e080 2023-11-24T06:26:08.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b677fe700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 0x7f2b68157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:08.107 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6cbd6700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2b60012000 tx=0x7f2b60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600048b0 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b600092e0 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6001c680 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.095+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b681544e0 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.096+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b68154aa0 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.096+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b68154c60 con 0x7f2b6815e080 2023-11-24T06:26:08.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.097+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2b6001c7e0 con 0x7f2b6815e080 2023-11-24T06:26:08.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.097+0000 7f2b64ff9700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 0x7f2b5406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.097+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b6008b230 con 0x7f2b6815e080 2023-11-24T06:26:08.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.097+0000 7f2b67fff700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 0x7f2b5406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.098+0000 7f2b67fff700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 0x7f2b5406d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b68156e80 tx=0x7f2b5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.112 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.102+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b60055fa0 con 0x7f2b6815e080 2023-11-24T06:26:08.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.307+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b680753f0 con 0x7f2b6815e080 2023-11-24T06:26:08.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.308+0000 7f2b64ff9700 1 -- 172.21.15.114:0/1551159864 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b600595c0 con 0x7f2b6815e080 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 msgr2=0x7f2b5406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 0x7f2b5406d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b68156e80 tx=0x7f2b5c008040 comp rx=0 tx=0).stop 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 msgr2=0x7f2b681f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2b60012000 tx=0x7f2b60004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 shutdown_connections 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f2b5406b080 0x7f2b5406d540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b6815e080 0x7f2b681f3a60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b6806c710 0x7f2b68153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 --2- 172.21.15.114:0/1551159864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b680af4d0 0x7f2b68157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 >> 172.21.15.114:0/1551159864 conn(0x7f2b68071b90 msgr2=0x7f2b680b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:08.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 shutdown_connections 2023-11-24T06:26:08.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.310+0000 7f2b6e639700 1 -- 172.21.15.114:0/1551159864 wait complete. 2023-11-24T06:26:08.315 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:26:08.326 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:26:08.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:08.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:26:08.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:26:08.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:26:08.370 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:26:08.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:08.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.548+0000 7f7408657700 1 Processor -- start 2023-11-24T06:26:08.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.548+0000 7f7408657700 1 -- start start 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7408657700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74000b2fc0 0x7f74000d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7408657700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74000d2d10 con 0x7f740015dfc0 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7408657700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74000cefd0 con 0x7f74000d1570 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7408657700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74000cf2b0 con 0x7f74000b2fc0 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7406bf4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74000b2fc0 0x7f74000d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f74063f3700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f740015dfc0 0x7f74000d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56972/0 (socket says 172.21.15.114:56972) 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7406bf4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74000b2fc0 0x7f74000d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41268/0 (socket says 172.21.15.114:41268) 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f74063f3700 1 -- 172.21.15.114:0/2789862103 learned_addr learned my addr 172.21.15.114:0/2789862103 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:08.554 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7406bf4700 1 -- 172.21.15.114:0/2789862103 >> v1:172.21.15.172:6789/0 conn(0x7f74000d1570 legacy=0x7f74000d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7406bf4700 1 -- 172.21.15.114:0/2789862103 >> v1:172.21.15.114:6789/0 conn(0x7f740015dfc0 legacy=0x7f74000d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.549+0000 7f7406bf4700 1 -- 172.21.15.114:0/2789862103 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000d2fb0 con 0x7f74000b2fc0 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7406bf4700 1 --2- 172.21.15.114:0/2789862103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74000b2fc0 0x7f74000d3be0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009d90 tx=0x7f73f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a4adefda9fe788 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f74053f1700 1 -- 172.21.15.114:0/2789862103 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800e070 con 0x7f74000b2fc0 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f74053f1700 1 -- 172.21.15.114:0/2789862103 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f800bb90 con 0x7f74000b2fc0 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f74053f1700 1 -- 172.21.15.114:0/2789862103 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8003a20 con 0x7f74000b2fc0 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 -- 172.21.15.114:0/2789862103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74000b2fc0 msgr2=0x7f74000d3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 --2- 172.21.15.114:0/2789862103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74000b2fc0 0x7f74000d3be0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009d90 tx=0x7f73f800b330 comp rx=0 tx=0).stop 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 -- 172.21.15.114:0/2789862103 shutdown_connections 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 --2- 172.21.15.114:0/2789862103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74000b2fc0 0x7f74000d3be0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 -- 172.21.15.114:0/2789862103 >> 172.21.15.114:0/2789862103 conn(0x7f740006b410 msgr2=0x7f740006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:08.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 -- 172.21.15.114:0/2789862103 shutdown_connections 2023-11-24T06:26:08.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.550+0000 7f7408657700 1 -- 172.21.15.114:0/2789862103 wait complete. 2023-11-24T06:26:08.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 Processor -- start 2023-11-24T06:26:08.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 -- start start 2023-11-24T06:26:08.559 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74000b2fc0 0x7f740015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74000d1570 0x7f74000ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740015aca0 con 0x7f740015dfc0 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74000cc830 con 0x7f74000d1570 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7408657700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74000c95c0 con 0x7f74000b2fc0 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7406bf4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7406bf4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41274/0 (socket says 172.21.15.114:41274) 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.552+0000 7f7406bf4700 1 -- 172.21.15.114:0/1467581830 learned_addr learned my addr 172.21.15.114:0/1467581830 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 -- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74000b2fc0 msgr2=0x7f740015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74000b2fc0 0x7f740015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 -- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74000d1570 msgr2=0x7f74000ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7405bf2700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74000d1570 0x7f74000ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74000d1570 0x7f74000ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 -- 172.21.15.114:0/1467581830 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f8009a70 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7406bf4700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f74000b5c40 tx=0x7f73f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8012630 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f80127b0 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74000c98a0 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8012b60 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.553+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74000c9e60 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.554+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74000c9ff0 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.554+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f73f8012cc0 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.555+0000 7f73f6ffd700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 0x7f73e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.555+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f73f808c090 con 0x7f740015dfc0 2023-11-24T06:26:08.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.555+0000 7f74063f3700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 0x7f73e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:08.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.555+0000 7f74063f3700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 0x7f73e406d540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f60 tx=0x7f73f0007b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:08.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.559+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73f8056e00 con 0x7f740015dfc0 2023-11-24T06:26:08.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.793+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7400067f60 con 0x7f740015dfc0 2023-11-24T06:26:08.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.794+0000 7f73f6ffd700 1 -- 172.21.15.114:0/1467581830 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f73f805a420 con 0x7f740015dfc0 2023-11-24T06:26:08.797 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:08.798 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 msgr2=0x7f73e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 0x7f73e406d540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f73f0000f60 tx=0x7f73f0007b10 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 msgr2=0x7f74000cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f74000b5c40 tx=0x7f73f8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 shutdown_connections 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f73e406b080 0x7f73e406d540 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f740015dfc0 0x7f74000cbd90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74000b2fc0 0x7f740015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 --2- 172.21.15.114:0/1467581830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74000d1570 0x7f74000ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:08.800 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 >> 172.21.15.114:0/1467581830 conn(0x7f740006b410 msgr2=0x7f74000d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:08.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 shutdown_connections 2023-11-24T06:26:08.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:08.796+0000 7f7408657700 1 -- 172.21.15.114:0/1467581830 wait complete. 2023-11-24T06:26:08.801 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:08.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:08.952 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2023-11-24T06:26:08.952 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-24T06:26:08.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6240 2023-11-24T06:26:09.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 Processor -- start 2023-11-24T06:26:09.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 -- start start 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f002c15e160 0x7f002c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 0x7f002c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 0x7f002c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c0b4500 con 0x7f002c15e160 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c074960 con 0x7f002c0744d0 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.073+0000 7f0032bd6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0b3c60 con 0x7f002c0afc90 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f002c15e160 0x7f002c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f002c15e160 0x7f002c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41278/0 (socket says 172.21.15.114:41278) 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 -- 172.21.15.114:0/650972389 learned_addr learned my addr 172.21.15.114:0/650972389 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0031173700 1 --2- 172.21.15.114:0/650972389 >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 0x7f002c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f002bfff700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 0x7f002c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 -- 172.21.15.114:0/650972389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 msgr2=0x7f002c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 0x7f002c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 -- 172.21.15.114:0/650972389 >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 msgr2=0x7f002c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 --2- 172.21.15.114:0/650972389 >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 0x7f002c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 -- 172.21.15.114:0/650972389 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0b58c0 con 0x7f002c15e160 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0031173700 1 --2- 172.21.15.114:0/650972389 >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 0x7f002c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f002bfff700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 0x7f002c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f0030972700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c15e160 0x7f002c0b5da0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f002c0d1cd0 tx=0x7f0020009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34facb34267b12ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f002b7fe700 1 -- 172.21.15.114:0/650972389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000b870 con 0x7f002c15e160 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f002b7fe700 1 -- 172.21.15.114:0/650972389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000b9d0 con 0x7f002c15e160 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.074+0000 7f002b7fe700 1 -- 172.21.15.114:0/650972389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000bd60 con 0x7f002c15e160 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 -- 172.21.15.114:0/650972389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c15e160 msgr2=0x7f002c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c15e160 0x7f002c0b5da0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f002c0d1cd0 tx=0x7f0020009bd0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 -- 172.21.15.114:0/650972389 shutdown_connections 2023-11-24T06:26:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 --2- 172.21.15.114:0/650972389 >> v2:172.21.15.172:3300/0 conn(0x7f002c0744d0 0x7f002c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c15e160 0x7f002c0b5da0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 --2- 172.21.15.114:0/650972389 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c0afc90 0x7f002c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 -- 172.21.15.114:0/650972389 >> 172.21.15.114:0/650972389 conn(0x7f002c0d0bf0 msgr2=0x7f002c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:09.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 -- 172.21.15.114:0/650972389 shutdown_connections 2023-11-24T06:26:09.081 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.075+0000 7f0032bd6700 1 -- 172.21.15.114:0/650972389 wait complete. 2023-11-24T06:26:09.083 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.076+0000 7f0032bd6700 1 Processor -- start 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.076+0000 7f0032bd6700 1 -- start start 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 0x7f002c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c15e160 0x7f002c160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c160140 con 0x7f002c0afc90 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0c5e70 con 0x7f002c0744d0 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0032bd6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c14eb00 con 0x7f002c15e160 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41286/0 (socket says 172.21.15.114:41286) 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 -- 172.21.15.114:0/2991779504 learned_addr learned my addr 172.21.15.114:0/2991779504 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0030972700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 0x7f002c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0031173700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c15e160 0x7f002c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 -- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c15e160 msgr2=0x7f002c160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c15e160 0x7f002c160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 -- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 msgr2=0x7f002c14e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 0x7f002c14e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 -- 172.21.15.114:0/2991779504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0020009550 con 0x7f002c0afc90 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f0030972700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 0x7f002c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.077+0000 7f002bfff700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f001800c750 tx=0x7f001800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.078+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018011800 con 0x7f002c0afc90 2023-11-24T06:26:09.085 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.078+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018011960 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.078+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c15f8a0 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.078+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018011cd0 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.078+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c15fe60 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.079+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c0c5bd0 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.079+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0018021350 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.079+0000 7f00297fa700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 0x7f001006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.080+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0018089950 con 0x7f002c0afc90 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.080+0000 7f0030972700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 0x7f001006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.086 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.080+0000 7f0030972700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 0x7f001006d540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0020014040 tx=0x7f0020008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.087 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.084+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0018054630 con 0x7f002c0afc90 2023-11-24T06:26:09.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.306+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6240", "target": ["mon-mgr", ""]}) v1 -- 0x7f002c0b4190 con 0x7f001006b080 2023-11-24T06:26:09.314 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:26:09.311+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:26:09.314 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:26:09.311+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:26:09.314 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:26:09.311+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:26:09.314 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:26:09.311+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:26:09.314 INFO:tasks.ceph.mgr.y.smithi114.stderr:2023-11-24T06:26:09.311+0000 7f60798dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:26:09.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.340+0000 7f00297fa700 1 -- 172.21.15.114:0/2991779504 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f002000b810 con 0x7f001006b080 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 msgr2=0x7f001006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 0x7f001006d540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0020014040 tx=0x7f0020008030 comp rx=0 tx=0).stop 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 msgr2=0x7f002c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f001800c750 tx=0x7f001800cb10 comp rx=0 tx=0).stop 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 shutdown_connections 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f001006b080 0x7f001006d540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f002c15e160 0x7f002c160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002c0744d0 0x7f002c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 --2- 172.21.15.114:0/2991779504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f002c0afc90 0x7f002c15ee00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 >> 172.21.15.114:0/2991779504 conn(0x7f002c0d0bf0 msgr2=0x7f002c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:09.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 shutdown_connections 2023-11-24T06:26:09.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.342+0000 7f0032bd6700 1 -- 172.21.15.114:0/2991779504 wait complete. 2023-11-24T06:26:09.358 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.474+0000 7ff694ed1700 1 Processor -- start 2023-11-24T06:26:09.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.474+0000 7ff694ed1700 1 -- start start 2023-11-24T06:26:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff694ed1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff6900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff694ed1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900ae430 con 0x7ff6900af4d0 2023-11-24T06:26:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff694ed1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900ae710 con 0x7ff69015e080 2023-11-24T06:26:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff694ed1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900af0b0 con 0x7ff69006c710 2023-11-24T06:26:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff68ed9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff6900af4d0 0x7ff69006e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57020/0 (socket says 172.21.15.114:57020) 2023-11-24T06:26:09.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.475+0000 7ff68ed9d700 1 -- 172.21.15.114:0/2775534820 learned_addr learned my addr 172.21.15.114:0/2775534820 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff6900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409034135 0 0) 0x7ff6900ae430 con 0x7ff6900af4d0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff67c0035b0 con 0x7ff6900af4d0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1482914926 0 0) 0x7ff67c0035b0 con 0x7ff6900af4d0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 msgr2=0x7ff6900b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff6900b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 >> v1:172.21.15.172:6789/0 conn(0x7ff69015e080 legacy=0x7ff69006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900ae9f0 con 0x7ff6900af4d0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7ff6880019f0 con 0x7ff6900af4d0 2023-11-24T06:26:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 >> v1:172.21.15.114:6789/0 conn(0x7ff6900af4d0 legacy=0x7ff69006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c003a30 0x7ff67c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c0043e0 0x7ff67c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff67c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900af0b0 con 0x7ff67c0043e0 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c00ba10 con 0x7ff67c003a30 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c00bbd0 con 0x7ff69006c710 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c0043e0 0x7ff67c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68e59c700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c003a30 0x7ff67c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68ed9d700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff67c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 -- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 msgr2=0x7ff67c005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff69006c710 0x7ff67c005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 -- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c003a30 msgr2=0x7ff67c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c003a30 0x7ff67c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.476+0000 7ff68dd9b700 1 -- 172.21.15.114:0/2775534820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0035b0 con 0x7ff67c0043e0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c0043e0 0x7ff67c0048b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff678009b60 tx=0x7ff67c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b499cd1ce878ad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800e6c0 con 0x7ff67c0043e0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800e820 con 0x7ff67c0043e0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff68d59a700 1 -- 172.21.15.114:0/2775534820 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800eb90 con 0x7ff67c0043e0 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 -- 172.21.15.114:0/2775534820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c0043e0 msgr2=0x7ff67c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 --2- 172.21.15.114:0/2775534820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c0043e0 0x7ff67c0048b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff678009b60 tx=0x7ff67c005880 comp rx=0 tx=0).stop 2023-11-24T06:26:09.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff68ed9d700 1 -- 172.21.15.114:0/2775534820 reap_dead start 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 -- 172.21.15.114:0/2775534820 shutdown_connections 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 -- 172.21.15.114:0/2775534820 >> 172.21.15.114:0/2775534820 conn(0x7ff690071b90 msgr2=0x7ff690071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 -- 172.21.15.114:0/2775534820 shutdown_connections 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.477+0000 7ff694ed1700 1 -- 172.21.15.114:0/2775534820 wait complete. 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.479+0000 7ff694ed1700 1 Processor -- start 2023-11-24T06:26:09.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.479+0000 7ff694ed1700 1 -- start start 2023-11-24T06:26:09.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.479+0000 7ff694ed1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0043e0 0x7ff69015ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff69014e810 0x7ff690151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015fb90 con 0x7ff69013e680 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69014ecc0 con 0x7ff69014e810 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015ef10 con 0x7ff67c0043e0 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41322/0 (socket says 172.21.15.114:41322) 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 -- 172.21.15.114:0/3522724227 learned_addr learned my addr 172.21.15.114:0/3522724227 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68e59c700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0043e0 0x7ff69015ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68ed9d700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff69014e810 0x7ff690151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 -- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0043e0 msgr2=0x7ff69015ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0043e0 0x7ff69015ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 -- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff69014e810 msgr2=0x7ff690151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff69014e810 0x7ff690151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 -- 172.21.15.114:0/3522724227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69015f1f0 con 0x7ff69013e680 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff68dd9b700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff67800e4b0 tx=0x7ff6780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800e6c0 con 0x7ff69013e680 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780038c0 con 0x7ff69013e680 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69015f4d0 con 0x7ff69013e680 2023-11-24T06:26:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678003c70 con 0x7ff69013e680 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.480+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6901516b0 con 0x7ff69013e680 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.481+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6900753f0 con 0x7ff69013e680 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.482+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff678013470 con 0x7ff69013e680 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.482+0000 7ff686ffd700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 0x7ff67006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.482+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff67808b170 con 0x7ff69013e680 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.482+0000 7ff68e59c700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 0x7ff67006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.482+0000 7ff68e59c700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 0x7ff67006d540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff6800010e0 tx=0x7ff680009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.486+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff678056110 con 0x7ff69013e680 2023-11-24T06:26:09.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.717+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff690067f60 con 0x7ff69013e680 2023-11-24T06:26:09.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.717+0000 7ff686ffd700 1 -- 172.21.15.114:0/3522724227 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff678059730 con 0x7ff69013e680 2023-11-24T06:26:09.721 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:09.721 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 msgr2=0x7ff67006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 0x7ff67006d540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff6800010e0 tx=0x7ff680009040 comp rx=0 tx=0).stop 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 msgr2=0x7ff69015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff67800e4b0 tx=0x7ff6780047c0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 shutdown_connections 2023-11-24T06:26:09.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff67006b080 0x7ff67006d540 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff69014e810 0x7ff690151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0043e0 0x7ff69015ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 --2- 172.21.15.114:0/3522724227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff69013e680 0x7ff69015e8c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 >> 172.21.15.114:0/3522724227 conn(0x7ff690071b90 msgr2=0x7ff6900af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 shutdown_connections 2023-11-24T06:26:09.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.719+0000 7ff694ed1700 1 -- 172.21.15.114:0/3522724227 wait complete. 2023-11-24T06:26:09.726 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:09.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:09.875 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.54798836399999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972693681454135,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:09.875 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.972693681454135 }, "state": "open", "uptime": 0.547988364 } ] 2023-11-24T06:26:09.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:09.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.993+0000 7f448991a700 1 Processor -- start 2023-11-24T06:26:09.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.993+0000 7f448991a700 1 -- start start 2023-11-24T06:26:09.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f448991a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f448406c710 0x7f44840b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f448991a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 0x7f44840afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:09.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f448991a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840ae430 con 0x7f448415e080 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f448991a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840af0b0 con 0x7f448406c710 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f448991a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840b05a0 con 0x7f44840af4d0 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f44827fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f448406c710 0x7f44840b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f44827fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f448406c710 0x7f44840b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41330/0 (socket says 172.21.15.114:41330) 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.994+0000 7f44827fc700 1 -- 172.21.15.114:0/1102545930 learned_addr learned my addr 172.21.15.114:0/1102545930 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44837fe700 1 --2- 172.21.15.114:0/1102545930 >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 0x7f44840afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44827fc700 1 -- 172.21.15.114:0/1102545930 >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 msgr2=0x7f44840afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44827fc700 1 --2- 172.21.15.114:0/1102545930 >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 0x7f44840afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44827fc700 1 -- 172.21.15.114:0/1102545930 >> v1:172.21.15.172:6789/0 conn(0x7f448415e080 legacy=0x7f448406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44827fc700 1 -- 172.21.15.114:0/1102545930 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840ae710 con 0x7f448406c710 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44837fe700 1 --2- 172.21.15.114:0/1102545930 >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 0x7f44840afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f44827fc700 1 --2- 172.21.15.114:0/1102545930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44840b5050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f448409eea0 tx=0x7f4474009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1510983b985735 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f4481ffb700 1 -- 172.21.15.114:0/1102545930 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474004230 con 0x7f448406c710 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f4481ffb700 1 -- 172.21.15.114:0/1102545930 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474004390 con 0x7f448406c710 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f4481ffb700 1 -- 172.21.15.114:0/1102545930 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740036a0 con 0x7f448406c710 2023-11-24T06:26:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 -- 172.21.15.114:0/1102545930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 msgr2=0x7f44840b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 --2- 172.21.15.114:0/1102545930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44840b5050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f448409eea0 tx=0x7f4474009c00 comp rx=0 tx=0).stop 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 -- 172.21.15.114:0/1102545930 shutdown_connections 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 --2- 172.21.15.114:0/1102545930 >> v2:172.21.15.172:3300/0 conn(0x7f44840af4d0 0x7f44840afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 --2- 172.21.15.114:0/1102545930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44840b5050 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.995+0000 7f448991a700 1 -- 172.21.15.114:0/1102545930 >> 172.21.15.114:0/1102545930 conn(0x7f4484071b90 msgr2=0x7f4484071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.996+0000 7f448991a700 1 -- 172.21.15.114:0/1102545930 shutdown_connections 2023-11-24T06:26:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.996+0000 7f448991a700 1 -- 172.21.15.114:0/1102545930 wait complete. 2023-11-24T06:26:10.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.997+0000 7f448991a700 1 Processor -- start 2023-11-24T06:26:10.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.997+0000 7f448991a700 1 -- start start 2023-11-24T06:26:10.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44840af4d0 0x7f4484148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f448415e080 0x7f448406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448415a4f0 con 0x7f448406c710 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484159970 con 0x7f44840af4d0 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4484159c50 con 0x7f448415e080 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41334/0 (socket says 172.21.15.114:41334) 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 -- 172.21.15.114:0/4116441042 learned_addr learned my addr 172.21.15.114:0/4116441042 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:10.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f44827fc700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44840af4d0 0x7f4484148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f44837fe700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f448415e080 0x7f448406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 -- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f448415e080 msgr2=0x7f448406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f448415e080 0x7f448406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 -- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44840af4d0 msgr2=0x7f4484148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44840af4d0 0x7f4484148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 -- 172.21.15.114:0/4116441042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009550 con 0x7f448406c710 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f4482ffd700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f447800b4a0 tx=0x7f447800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478011830 con 0x7f448406c710 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4478011990 con 0x7f448406c710 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484159f30 con 0x7f448406c710 2023-11-24T06:26:10.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478011d40 con 0x7f448406c710 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.998+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484157620 con 0x7f448406c710 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:09.999+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44840753f0 con 0x7f448406c710 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.000+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 6) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f447801a400 con 0x7f448406c710 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.000+0000 7f446bfff700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 0x7f447006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.000+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4478089480 con 0x7f448406c710 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.000+0000 7f44827fc700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 0x7f447006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:10.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.001+0000 7f44827fc700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 0x7f447006d540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4474009130 tx=0x7f4474008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:10.009 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.005+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4478054300 con 0x7f448406c710 2023-11-24T06:26:10.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.237+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4484067f60 con 0x7f448406c710 2023-11-24T06:26:10.243 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.238+0000 7f446bfff700 1 -- 172.21.15.114:0/4116441042 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f4478057920 con 0x7f448406c710 2023-11-24T06:26:10.245 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:10.245 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 msgr2=0x7f447006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 0x7f447006d540 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4474009130 tx=0x7f4474008030 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 msgr2=0x7f44841570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f447800b4a0 tx=0x7f447800b7b0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 shutdown_connections 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f447006b080 0x7f447006d540 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f448415e080 0x7f448406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f448406c710 0x7f44841570e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 --2- 172.21.15.114:0/4116441042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44840af4d0 0x7f4484148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:10.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 >> 172.21.15.114:0/4116441042 conn(0x7f4484071b90 msgr2=0x7f44840b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:10.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 shutdown_connections 2023-11-24T06:26:10.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:10.240+0000 7f448991a700 1 -- 172.21.15.114:0/4116441042 wait complete. 2023-11-24T06:26:10.247 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:10.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:10.396 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.068977302,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.946260182897081,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:10.396 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.94626018289708 }, "state": "open", "uptime": 1.068977302 } ] 2023-11-24T06:26:10.396 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:15.398 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:15.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.527+0000 7f201a865700 1 Processor -- start 2023-11-24T06:26:15.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.527+0000 7f201a865700 1 -- start start 2023-11-24T06:26:15.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f201a865700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20141604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f201a865700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20140d1c70 0x7f20140d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f201a865700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20140b22d0 con 0x7f20140b3370 2023-11-24T06:26:15.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f201a865700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20140b2f50 con 0x7f20140d1c70 2023-11-24T06:26:15.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f201a865700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20140d2640 con 0x7f201415bdd0 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f20137fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f20140b3370 0x7f20140b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57050/0 (socket says 172.21.15.114:57050) 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f20137fe700 1 -- 172.21.15.114:0/943036807 learned_addr learned my addr 172.21.15.114:0/943036807 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2018e02700 1 --2- 172.21.15.114:0/943036807 >> v2:172.21.15.114:3300/0 conn(0x7f20140d1c70 0x7f20140d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2013fff700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20141604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2012ffd700 1 -- 172.21.15.114:0/943036807 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209513439 0 0) 0x7f20140b22d0 con 0x7f20140b3370 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2012ffd700 1 -- 172.21.15.114:0/943036807 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ffc0035b0 con 0x7f20140b3370 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2018e02700 1 -- 172.21.15.114:0/943036807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 msgr2=0x7f20141604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.528+0000 7f2018e02700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20141604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2018e02700 1 -- 172.21.15.114:0/943036807 >> v1:172.21.15.114:6789/0 conn(0x7f20140b3370 legacy=0x7f20140b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2018e02700 1 -- 172.21.15.114:0/943036807 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20140b25b0 con 0x7f20140d1c70 2023-11-24T06:26:15.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2018e02700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140d2100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a2f1baf8cf471f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2012ffd700 1 -- 172.21.15.114:0/943036807 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c00e070 con 0x7f20140d1c70 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2012ffd700 1 -- 172.21.15.114:0/943036807 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200c00bb20 con 0x7f20140d1c70 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f2012ffd700 1 -- 172.21.15.114:0/943036807 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c012430 con 0x7f20140d1c70 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 -- 172.21.15.114:0/943036807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 msgr2=0x7f20140d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140d2100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f200c009d90 tx=0x7f200c00b330 comp rx=0 tx=0).stop 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 -- 172.21.15.114:0/943036807 shutdown_connections 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20141604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 --2- 172.21.15.114:0/943036807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140d2100 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.529+0000 7f201a865700 1 -- 172.21.15.114:0/943036807 >> 172.21.15.114:0/943036807 conn(0x7f20140731a0 msgr2=0x7f201409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:15.535 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.530+0000 7f201a865700 1 -- 172.21.15.114:0/943036807 shutdown_connections 2023-11-24T06:26:15.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.530+0000 7f201a865700 1 -- 172.21.15.114:0/943036807 wait complete. 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.531+0000 7f201a865700 1 Processor -- start 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.531+0000 7f201a865700 1 -- start start 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 0x7f2014152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20140c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c99c0 con 0x7f20140d1c70 2023-11-24T06:26:15.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014152850 con 0x7f20140b3370 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c8db0 con 0x7f201415bdd0 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41350/0 (socket says 172.21.15.114:41350) 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 -- 172.21.15.114:0/3117876628 learned_addr learned my addr 172.21.15.114:0/3117876628 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2013fff700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 0x7f2014152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2018e02700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20140c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 -- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 msgr2=0x7f20140c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20140c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 -- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 msgr2=0x7f2014152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 0x7f2014152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 -- 172.21.15.114:0/3117876628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c009a70 con 0x7f20140d1c70 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2013fff700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 0x7f2014152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:15.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f20137fe700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f200400c700 tx=0x7f200400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004005d70 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20040070d0 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004007460 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.532+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20140c9090 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.533+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20140c9650 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.533+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20140c9ca0 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.534+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f20040075c0 con 0x7f20140d1c70 2023-11-24T06:26:15.542 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.534+0000 7f2010ff9700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 0x7f200006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:15.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.534+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2004089060 con 0x7f20140d1c70 2023-11-24T06:26:15.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.534+0000 7f2013fff700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 0x7f200006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:15.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.535+0000 7f2013fff700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 0x7f200006d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2008009b90 tx=0x7f2008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:15.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.539+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2004053d80 con 0x7f20140d1c70 2023-11-24T06:26:15.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.774+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20140d01b0 con 0x7f20140d1c70 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.775+0000 7f2010ff9700 1 -- 172.21.15.114:0/3117876628 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f20040573a0 con 0x7f20140d1c70 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:15.778 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:15.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 msgr2=0x7f200006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 0x7f200006d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2008009b90 tx=0x7f2008008040 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 msgr2=0x7f20140c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f200400c700 tx=0x7f200400cac0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 shutdown_connections 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f200006b110 0x7f200006d5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f201415bdd0 0x7f20140c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20140b3370 0x7f2014152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 --2- 172.21.15.114:0/3117876628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20140d1c70 0x7f20140c6200 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 >> 172.21.15.114:0/3117876628 conn(0x7f20140731a0 msgr2=0x7f20140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 shutdown_connections 2023-11-24T06:26:15.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:15.777+0000 7f201a865700 1 -- 172.21.15.114:0/3117876628 wait complete. 2023-11-24T06:26:15.782 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:15.792 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:15.932 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.6048597559999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.667341820988735,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:15.933 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.667341820988735 }, "state": "open", "uptime": 6.604859756 } ] 2023-11-24T06:26:15.933 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:16.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:16.497 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:20.934 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:21.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.064+0000 7f0ffa14a700 1 Processor -- start 2023-11-24T06:26:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.064+0000 7f0ffa14a700 1 -- start start 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ffa14a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0ff415e080 0x7f0ff40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ffa14a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ffa14a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40ae430 con 0x7f0ff40af4d0 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ffa14a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40af0b0 con 0x7f0ff415e080 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ffa14a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b05a0 con 0x7f0ff406c710 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0ff415e080 0x7f0ff40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0ff415e080 0x7f0ff40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41356/0 (socket says 172.21.15.114:41356) 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 -- 172.21.15.114:0/2722315693 learned_addr learned my addr 172.21.15.114:0/2722315693 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff3fff700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 -- 172.21.15.114:0/2722315693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 msgr2=0x7f0ff40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 -- 172.21.15.114:0/2722315693 >> v1:172.21.15.172:6789/0 conn(0x7f0ff40af4d0 legacy=0x7f0ff406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff37fe700 1 -- 172.21.15.114:0/2722315693 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40ae710 con 0x7f0ff415e080 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.065+0000 7f0ff3fff700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff415e080 0x7f0ff40b5050 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009920 tx=0x7f0fdc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a71ccd180b517e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ff27fc700 1 -- 172.21.15.114:0/2722315693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc004230 con 0x7f0ff415e080 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ff27fc700 1 -- 172.21.15.114:0/2722315693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc004390 con 0x7f0ff415e080 2023-11-24T06:26:21.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ff27fc700 1 -- 172.21.15.114:0/2722315693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc009e70 con 0x7f0ff415e080 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2722315693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff415e080 msgr2=0x7f0ff40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff415e080 0x7f0ff40b5050 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009920 tx=0x7f0fdc009c30 comp rx=0 tx=0).stop 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2722315693 shutdown_connections 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2722315693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff415e080 0x7f0ff40b5050 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2722315693 >> 172.21.15.114:0/2722315693 conn(0x7f0ff4071b90 msgr2=0x7f0ff4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2722315693 shutdown_connections 2023-11-24T06:26:21.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.066+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2722315693 wait complete. 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 Processor -- start 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 -- start start 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 0x7f0ff4157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4154440 con 0x7f0ff40af4d0 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41534e0 con 0x7f0ff415e080 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.068+0000 7f0ffa14a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41f8330 con 0x7f0ff406c710 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41364/0 (socket says 172.21.15.114:41364) 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 -- 172.21.15.114:0/2799113666 learned_addr learned my addr 172.21.15.114:0/2799113666 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff3fff700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 0x7f0ff4157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 -- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 msgr2=0x7f0ff4151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff4151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 -- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 msgr2=0x7f0ff4157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 0x7f0ff4157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 -- 172.21.15.114:0/2799113666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009550 con 0x7f0ff40af4d0 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff3fff700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 0x7f0ff4157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ff2ffd700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400c4a0 tx=0x7f0fe400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:21.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4011790 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40118f0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4011ca0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff4153d80 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.069+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff4152510 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.070+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff40753f0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.070+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0fe401c460 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.071+0000 7f0febfff700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 0x7f0fd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.071+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fe40897d0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.071+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 0x7f0fd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:21.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.071+0000 7f0ff37fe700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 0x7f0fd406d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009c70 tx=0x7f0fdc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:21.078 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.076+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fe40545b0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.305+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ff40d09b0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.306+0000 7f0febfff700 1 -- 172.21.15.114:0/2799113666 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f0fe4057bd0 con 0x7f0ff40af4d0 2023-11-24T06:26:21.308 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:21.308 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:21.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 msgr2=0x7f0fd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:21.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 0x7f0fd406d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009c70 tx=0x7f0fdc008030 comp rx=0 tx=0).stop 2023-11-24T06:26:21.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 msgr2=0x7f0ff4145440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400c4a0 tx=0x7f0fe400c860 comp rx=0 tx=0).stop 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 shutdown_connections 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0fd406b110 0x7f0fd406d5d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff415e080 0x7f0ff4157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff406c710 0x7f0ff4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 --2- 172.21.15.114:0/2799113666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff40af4d0 0x7f0ff4145440 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 >> 172.21.15.114:0/2799113666 conn(0x7f0ff4071b90 msgr2=0x7f0ff40b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 shutdown_connections 2023-11-24T06:26:21.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:21.308+0000 7f0ffa14a700 1 -- 172.21.15.114:0/2799113666 wait complete. 2023-11-24T06:26:21.312 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:21.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:21.462 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.135742386,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.39221539744295,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:21.463 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.39221539744295 }, "state": "open", "uptime": 12.135742386 } ] 2023-11-24T06:26:21.463 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:26.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:26.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.596+0000 7fa737c7d700 1 Processor -- start 2023-11-24T06:26:26.598 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.596+0000 7fa737c7d700 1 -- start start 2023-11-24T06:26:26.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa737c7d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa737c7d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa73015de50 0x7fa73015e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa737c7d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300cc5c0 con 0x7fa730151500 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa737c7d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa730071d70 con 0x7fa73015de50 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa737c7d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015e820 con 0x7fa7300b49b0 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa73621a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa73015de50 0x7fa73015e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa735218700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa735218700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:35914/0 (socket says 172.21.15.114:35914) 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa735a19700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa730151500 0x7fa7300cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:50674/0 (socket says 172.21.15.114:50674) 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.597+0000 7fa735218700 1 -- 172.21.15.114:0/2993296756 learned_addr learned my addr 172.21.15.114:0/2993296756 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa735218700 1 -- 172.21.15.114:0/2993296756 >> v2:172.21.15.172:3300/0 conn(0x7fa73015de50 msgr2=0x7fa73015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa735218700 1 --2- 172.21.15.114:0/2993296756 >> v2:172.21.15.172:3300/0 conn(0x7fa73015de50 0x7fa73015e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa734a17700 1 -- 172.21.15.114:0/2993296756 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106184434 0 0) 0x7fa7300cc5c0 con 0x7fa730151500 2023-11-24T06:26:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa735218700 1 -- 172.21.15.114:0/2993296756 >> v1:172.21.15.172:6789/0 conn(0x7fa730151500 legacy=0x7fa7300cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa735218700 1 -- 172.21.15.114:0/2993296756 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300717b0 con 0x7fa7300b49b0 2023-11-24T06:26:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa735218700 1 --2- 172.21.15.114:0/2993296756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa72000aa90 tx=0x7fa72000ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b554c648cbfb734b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa734a17700 1 -- 172.21.15.114:0/2993296756 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000cb70 con 0x7fa7300b49b0 2023-11-24T06:26:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa734a17700 1 -- 172.21.15.114:0/2993296756 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72000ccd0 con 0x7fa7300b49b0 2023-11-24T06:26:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.598+0000 7fa734a17700 1 -- 172.21.15.114:0/2993296756 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720018620 con 0x7fa7300b49b0 2023-11-24T06:26:26.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 -- 172.21.15.114:0/2993296756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 msgr2=0x7fa73006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2993296756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa72000aa90 tx=0x7fa72000ada0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 -- 172.21.15.114:0/2993296756 shutdown_connections 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2993296756 >> v2:172.21.15.172:3300/0 conn(0x7fa73015de50 0x7fa73015e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2993296756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7300b49b0 0x7fa73006d350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 -- 172.21.15.114:0/2993296756 >> 172.21.15.114:0/2993296756 conn(0x7fa7300b1740 msgr2=0x7fa7300b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 -- 172.21.15.114:0/2993296756 shutdown_connections 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.599+0000 7fa737c7d700 1 -- 172.21.15.114:0/2993296756 wait complete. 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.600+0000 7fa737c7d700 1 Processor -- start 2023-11-24T06:26:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.600+0000 7fa737c7d700 1 -- start start 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7300b49b0 0x7fa73013b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa73015de50 0x7fa7301575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015c8f0 con 0x7fa730151500 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015d150 con 0x7fa7300b49b0 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa737c7d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015cb90 con 0x7fa73015de50 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.601+0000 7fa735218700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41378/0 (socket says 172.21.15.114:41378) 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 -- 172.21.15.114:0/2500214821 learned_addr learned my addr 172.21.15.114:0/2500214821 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735a19700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7300b49b0 0x7fa73013b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa73621a700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa73015de50 0x7fa7301575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 -- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa73015de50 msgr2=0x7fa7301575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa73015de50 0x7fa7301575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 -- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7300b49b0 msgr2=0x7fa73013b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7300b49b0 0x7fa73013b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 -- 172.21.15.114:0/2500214821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72000a6f0 con 0x7fa730151500 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa735218700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c4d0 tx=0x7fa7200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720003ab0 con 0x7fa730151500 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720003c10 con 0x7fa730151500 2023-11-24T06:26:26.608 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73015ce70 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720020400 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.602+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa730151ec0 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.603+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7300751e0 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.603+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa720018400 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.604+0000 7fa7267fc700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 0x7fa71406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.604+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa72008b150 con 0x7fa730151500 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.604+0000 7fa735a19700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 0x7fa71406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.604+0000 7fa735a19700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 0x7fa71406d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000f60 tx=0x7fa72c007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:26.611 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.608+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa720055cc0 con 0x7fa730151500 2023-11-24T06:26:26.846 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.844+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73015d820 con 0x7fa730151500 2023-11-24T06:26:26.847 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.844+0000 7fa7267fc700 1 -- 172.21.15.114:0/2500214821 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fa7200592e0 con 0x7fa730151500 2023-11-24T06:26:26.847 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:26.847 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:26.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 msgr2=0x7fa71406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 0x7fa71406d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000f60 tx=0x7fa72c007be0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.850 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 msgr2=0x7fa730151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c4d0 tx=0x7fa7200047c0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 shutdown_connections 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa71406b110 0x7fa71406d5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa73015de50 0x7fa7301575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7300b49b0 0x7fa73013b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 --2- 172.21.15.114:0/2500214821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa730151500 0x7fa730151980 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 >> 172.21.15.114:0/2500214821 conn(0x7fa7300b1740 msgr2=0x7fa7300b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 shutdown_connections 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:26.847+0000 7fa737c7d700 1 -- 172.21.15.114:0/2500214821 wait complete. 2023-11-24T06:26:26.851 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:26.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:27.002 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.676624919999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.120094480852138,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:27.003 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.12009448085214 }, "state": "open", "uptime": 17.67662492 } ] 2023-11-24T06:26:27.003 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:32.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:32.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.137+0000 7f9360d86700 1 Processor -- start 2023-11-24T06:26:32.139 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.137+0000 7f9360d86700 1 -- start start 2023-11-24T06:26:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9360d86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9360d86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f935c0d2240 0x7f935c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9360d86700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0b1d40 con 0x7f935c15e1b0 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9360d86700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c0b28c0 con 0x7f935c0d2240 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9360d86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c06c000 con 0x7f935c15ddd0 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f935c0d2240 0x7f935c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9359d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f935c15e1b0 0x7f935c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57094/0 (socket says 172.21.15.114:57094) 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f9359d9b700 1 -- 172.21.15.114:0/860254571 learned_addr learned my addr 172.21.15.114:0/860254571 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935a59c700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 -- 172.21.15.114:0/860254571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 msgr2=0x7f935c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 -- 172.21.15.114:0/860254571 >> v1:172.21.15.114:6789/0 conn(0x7f935c15e1b0 legacy=0x7f935c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 -- 172.21.15.114:0/860254571 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0b1f90 con 0x7f935c0d2240 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935959a700 1 -- 172.21.15.114:0/860254571 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722403602 0 0) 0x7f935c0b1d40 con 0x7f935c15e1b0 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935a59c700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935ad9d700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c0d2240 0x7f935c0cf000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9354009d90 tx=0x7f935400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b58de9a819b1e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935959a700 1 -- 172.21.15.114:0/860254571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400e070 con 0x7f935c0d2240 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.138+0000 7f935959a700 1 -- 172.21.15.114:0/860254571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935400bbd0 con 0x7f935c0d2240 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f935959a700 1 -- 172.21.15.114:0/860254571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354012430 con 0x7f935c0d2240 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 -- 172.21.15.114:0/860254571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c0d2240 msgr2=0x7f935c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c0d2240 0x7f935c0cf000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9354009d90 tx=0x7f935400b330 comp rx=0 tx=0).stop 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 -- 172.21.15.114:0/860254571 shutdown_connections 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 --2- 172.21.15.114:0/860254571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c0d2240 0x7f935c0cf000 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 -- 172.21.15.114:0/860254571 >> 172.21.15.114:0/860254571 conn(0x7f935c0731a0 msgr2=0x7f935c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:32.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 -- 172.21.15.114:0/860254571 shutdown_connections 2023-11-24T06:26:32.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.139+0000 7f9360d86700 1 -- 172.21.15.114:0/860254571 wait complete. 2023-11-24T06:26:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.140+0000 7f9360d86700 1 Processor -- start 2023-11-24T06:26:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 -- start start 2023-11-24T06:26:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 0x7f935c0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c06f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09c110 con 0x7f935c15e1b0 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c06fc50 con 0x7f935c0d2240 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f9360d86700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09bb00 con 0x7f935c15ddd0 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f935ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f935ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41394/0 (socket says 172.21.15.114:41394) 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.141+0000 7f935ad9d700 1 -- 172.21.15.114:0/3152410756 learned_addr learned my addr 172.21.15.114:0/3152410756 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9359d9b700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935a59c700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 0x7f935c0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 -- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 msgr2=0x7f935c06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 -- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 msgr2=0x7f935c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 0x7f935c0b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 -- 172.21.15.114:0/3152410756 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354009a70 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9359d9b700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935a59c700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 0x7f935c0b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f935ad9d700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f935c0716d0 tx=0x7f9354004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354003ea0 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354020070 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0b2ed0 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354004280 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.142+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c0701b0 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.143+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c067f60 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.143+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f93540043e0 con 0x7f935c15e1b0 2023-11-24T06:26:32.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.144+0000 7f9352ffd700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 0x7f933c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.144+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9354022070 con 0x7f935c15e1b0 2023-11-24T06:26:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.144+0000 7f935a59c700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 0x7f933c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.144+0000 7f935a59c700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 0x7f933c06d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f93440089b0 tx=0x7f9344008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.148+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9354056a30 con 0x7f935c15e1b0 2023-11-24T06:26:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.380+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f935c0b7c60 con 0x7f935c15e1b0 2023-11-24T06:26:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.381+0000 7f9352ffd700 1 -- 172.21.15.114:0/3152410756 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f935401ae60 con 0x7f935c15e1b0 2023-11-24T06:26:32.383 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:32.384 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 msgr2=0x7f933c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 0x7f933c06d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f93440089b0 tx=0x7f9344008040 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 msgr2=0x7f935c1f2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f935c0716d0 tx=0x7f9354004ad0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 shutdown_connections 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f933c06b110 0x7f933c06d5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f935c15e1b0 0x7f935c1f2ee0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f935c0d2240 0x7f935c0b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 --2- 172.21.15.114:0/3152410756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f935c15ddd0 0x7f935c06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 >> 172.21.15.114:0/3152410756 conn(0x7f935c0731a0 msgr2=0x7f935c1606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 shutdown_connections 2023-11-24T06:26:32.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:32.383+0000 7f9360d86700 1 -- 172.21.15.114:0/3152410756 wait complete. 2023-11-24T06:26:32.387 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:32.398 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:32.537 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.210507603,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.851772329024719,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:32.538 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.85177232902472 }, "state": "open", "uptime": 23.210507603 } ] 2023-11-24T06:26:32.538 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:37.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.669+0000 7ff03b052700 1 Processor -- start 2023-11-24T06:26:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.669+0000 7ff03b052700 1 -- start start 2023-11-24T06:26:37.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff03b052700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 0x7ff0340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:37.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff03b052700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340ae430 con 0x7ff03415e080 2023-11-24T06:26:37.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff03b052700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340ae710 con 0x7ff0340af4d0 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff03b052700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340af0b0 con 0x7ff03406c710 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff038dee700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff03415e080 0x7ff03406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57106/0 (socket says 172.21.15.114:57106) 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff038dee700 1 -- 172.21.15.114:0/372275501 learned_addr learned my addr 172.21.15.114:0/372275501 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff033fff700 1 --2- 172.21.15.114:0/372275501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 0x7ff0340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616076296 0 0) 0x7ff0340ae430 con 0x7ff03415e080 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.670+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0200035b0 con 0x7ff03415e080 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725170111 0 0) 0x7ff0340ae710 con 0x7ff0340af4d0 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0340ae430 con 0x7ff0340af4d0 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1434618973 0 0) 0x7ff0200035b0 con 0x7ff03415e080 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff033fff700 1 -- 172.21.15.114:0/372275501 >> v1:172.21.15.172:6789/0 conn(0x7ff0340af4d0 legacy=0x7ff03406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff033fff700 1 -- 172.21.15.114:0/372275501 >> v1:172.21.15.114:6789/0 conn(0x7ff03415e080 legacy=0x7ff03406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff033fff700 1 -- 172.21.15.114:0/372275501 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340ae9f0 con 0x7ff03406c710 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff033fff700 1 --2- 172.21.15.114:0/372275501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 0x7ff0340b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009870 tx=0x7ff01c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6daf6440f501f68b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0019f0 con 0x7ff03406c710 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c004030 con 0x7ff03406c710 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.671+0000 7ff0337fe700 1 -- 172.21.15.114:0/372275501 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0043c0 con 0x7ff03406c710 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 -- 172.21.15.114:0/372275501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 msgr2=0x7ff0340b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 --2- 172.21.15.114:0/372275501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 0x7ff0340b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009870 tx=0x7ff01c009b80 comp rx=0 tx=0).stop 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 -- 172.21.15.114:0/372275501 shutdown_connections 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 --2- 172.21.15.114:0/372275501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03406c710 0x7ff0340b5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 -- 172.21.15.114:0/372275501 >> 172.21.15.114:0/372275501 conn(0x7ff034071b90 msgr2=0x7ff034071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 -- 172.21.15.114:0/372275501 shutdown_connections 2023-11-24T06:26:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.672+0000 7ff03b052700 1 -- 172.21.15.114:0/372275501 wait complete. 2023-11-24T06:26:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.673+0000 7ff03b052700 1 Processor -- start 2023-11-24T06:26:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.673+0000 7ff03b052700 1 -- start start 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 0x7ff034144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 0x7ff034160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034145370 con 0x7ff03406c710 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034155f50 con 0x7ff0340af4d0 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff03b052700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff034156230 con 0x7ff03415e080 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41404/0 (socket says 172.21.15.114:41404) 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 -- 172.21.15.114:0/263747707 learned_addr learned my addr 172.21.15.114:0/263747707 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff0395ef700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 0x7ff034160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff033fff700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 0x7ff034144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 -- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 msgr2=0x7ff034160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 0x7ff034160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 -- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 msgr2=0x7ff034144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 0x7ff034144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 -- 172.21.15.114:0/263747707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c0094a0 con 0x7ff03406c710 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff0395ef700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 0x7ff034160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff033fff700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 0x7ff034144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.674+0000 7ff038dee700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00cc70 tx=0x7ff02c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.675+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c00ade0 con 0x7ff03406c710 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.675+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034156510 con 0x7ff03406c710 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.675+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00ce70 con 0x7ff03406c710 2023-11-24T06:26:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.675+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0072f0 con 0x7ff03406c710 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.675+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034156ad0 con 0x7ff03406c710 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.676+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034156c90 con 0x7ff03406c710 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.676+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff02c007450 con 0x7ff03406c710 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.676+0000 7ff0317fa700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 0x7ff01406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.676+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff02c089670 con 0x7ff03406c710 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.676+0000 7ff033fff700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 0x7ff01406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:37.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.677+0000 7ff033fff700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 0x7ff01406d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0090b0 tx=0x7ff01c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:37.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.681+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff02c054300 con 0x7ff03406c710 2023-11-24T06:26:37.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.917+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0340753f0 con 0x7ff03406c710 2023-11-24T06:26:37.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.917+0000 7ff0317fa700 1 -- 172.21.15.114:0/263747707 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff02c057920 con 0x7ff03406c710 2023-11-24T06:26:37.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:37.920 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:37.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.919+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 msgr2=0x7ff01406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.919+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 0x7ff01406d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0090b0 tx=0x7ff01c008030 comp rx=0 tx=0).stop 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.919+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 msgr2=0x7ff034155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.919+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00cc70 tx=0x7ff02c00a340 comp rx=0 tx=0).stop 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 shutdown_connections 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff01406b110 0x7ff01406d5d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff03415e080 0x7ff034160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff03406c710 0x7ff034155870 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 --2- 172.21.15.114:0/263747707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0340af4d0 0x7ff034144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 >> 172.21.15.114:0/263747707 conn(0x7ff034071b90 msgr2=0x7ff0340b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:37.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 shutdown_connections 2023-11-24T06:26:37.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:37.920+0000 7ff03b052700 1 -- 172.21.15.114:0/263747707 wait complete. 2023-11-24T06:26:37.925 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:37.934 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:38.074 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.748390202,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.586668842156755,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:38.075 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.586668842156755 }, "state": "open", "uptime": 28.748390202 } ] 2023-11-24T06:26:38.076 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:43.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:43.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.208+0000 7f164e3df700 1 Processor -- start 2023-11-24T06:26:43.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.208+0000 7f164e3df700 1 -- start start 2023-11-24T06:26:43.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164e3df700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 0x7f16480b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:43.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164e3df700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164815f520 con 0x7f164806c710 2023-11-24T06:26:43.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164e3df700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164815f800 con 0x7f164815ea60 2023-11-24T06:26:43.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164e3df700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480b4230 con 0x7f1648144660 2023-11-24T06:26:43.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164c97c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f164806c710 0x7f164806ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57124/0 (socket says 172.21.15.114:57124) 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f164c97c700 1 -- 172.21.15.114:0/1571820013 learned_addr learned my addr 172.21.15.114:0/1571820013 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f1647fff700 1 --2- 172.21.15.114:0/1571820013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 0x7f16480b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.209+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18330828 0 0) 0x7f164815f520 con 0x7f164806c710 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16300035b0 con 0x7f164806c710 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346493240 0 0) 0x7f164815f800 con 0x7f164815ea60 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164815f520 con 0x7f164815ea60 2023-11-24T06:26:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960755373 0 0) 0x7f16300035b0 con 0x7f164806c710 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1647fff700 1 -- 172.21.15.114:0/1571820013 >> v1:172.21.15.172:6789/0 conn(0x7f164815ea60 legacy=0x7f164806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1647fff700 1 -- 172.21.15.114:0/1571820013 >> v1:172.21.15.114:6789/0 conn(0x7f164806c710 legacy=0x7f164806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1647fff700 1 -- 172.21.15.114:0/1571820013 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16480b3e50 con 0x7f1648144660 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1647fff700 1 --2- 172.21.15.114:0/1571820013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 0x7f16480b4650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1638009870 tx=0x7f1638009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e8864efdf2fef9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16400019f0 con 0x7f1648144660 2023-11-24T06:26:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638004030 con 0x7f1648144660 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.210+0000 7f1646ffd700 1 -- 172.21.15.114:0/1571820013 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16380043a0 con 0x7f1648144660 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 -- 172.21.15.114:0/1571820013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 msgr2=0x7f16480b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 --2- 172.21.15.114:0/1571820013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 0x7f16480b4650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1638009870 tx=0x7f1638009b80 comp rx=0 tx=0).stop 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 -- 172.21.15.114:0/1571820013 shutdown_connections 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 --2- 172.21.15.114:0/1571820013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1648144660 0x7f16480b4650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 -- 172.21.15.114:0/1571820013 >> 172.21.15.114:0/1571820013 conn(0x7f1648071ac0 msgr2=0x7f1648071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:43.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 -- 172.21.15.114:0/1571820013 shutdown_connections 2023-11-24T06:26:43.219 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.211+0000 7f164e3df700 1 -- 172.21.15.114:0/1571820013 wait complete. 2023-11-24T06:26:43.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.212+0000 7f164e3df700 1 Processor -- start 2023-11-24T06:26:43.221 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.212+0000 7f164e3df700 1 -- start start 2023-11-24T06:26:43.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:43.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 0x7f16481f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:43.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 0x7f16480ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:43.222 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480ae9b0 con 0x7f164806c710 2023-11-24T06:26:43.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16481f2840 con 0x7f1648144660 2023-11-24T06:26:43.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164e3df700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480ad4f0 con 0x7f164815ea60 2023-11-24T06:26:43.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:43.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41418/0 (socket says 172.21.15.114:41418) 2023-11-24T06:26:43.223 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 -- 172.21.15.114:0/1716726552 learned_addr learned my addr 172.21.15.114:0/1716726552 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:43.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164c97c700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 0x7f16480ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:43.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 -- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 msgr2=0x7f16480ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f16477fe700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 0x7f16481f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:43.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 0x7f16480ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.224 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 -- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 msgr2=0x7f16481f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 0x7f16481f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f1647fff700 1 -- 172.21.15.114:0/1716726552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16380094a0 con 0x7f164806c710 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f164c97c700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 0x7f16480ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.213+0000 7f16477fe700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 0x7f16481f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f1647fff700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1638000c00 tx=0x7f1638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638003e40 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163801f070 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16480aec50 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638011870 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.214+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16480af210 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.215+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648067f60 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.215+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f16380119d0 con 0x7f164806c710 2023-11-24T06:26:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.216+0000 7f1644ff9700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 0x7f163406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:43.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.216+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1638021070 con 0x7f164806c710 2023-11-24T06:26:43.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.216+0000 7f16477fe700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 0x7f163406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:43.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.216+0000 7f16477fe700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 0x7f163406d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f164813fb30 tx=0x7f163c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:43.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.220+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1638056190 con 0x7f164806c710 2023-11-24T06:26:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.452+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16480cd3d0 con 0x7f164806c710 2023-11-24T06:26:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.453+0000 7f1644ff9700 1 -- 172.21.15.114:0/1716726552 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f16380597b0 con 0x7f164806c710 2023-11-24T06:26:43.456 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:43.456 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:43.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 msgr2=0x7f163406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 0x7f163406d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f164813fb30 tx=0x7f163c009040 comp rx=0 tx=0).stop 2023-11-24T06:26:43.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 msgr2=0x7f16481f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:43.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1638000c00 tx=0x7f1638004750 comp rx=0 tx=0).stop 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 shutdown_connections 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f163406b110 0x7f163406d5d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f164815ea60 0x7f16480ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f164806c710 0x7f16481f2300 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 --2- 172.21.15.114:0/1716726552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1648144660 0x7f16481f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 >> 172.21.15.114:0/1716726552 conn(0x7f1648071ac0 msgr2=0x7f164806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 shutdown_connections 2023-11-24T06:26:43.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:43.455+0000 7f164e3df700 1 -- 172.21.15.114:0/1716726552 wait complete. 2023-11-24T06:26:43.460 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:43.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:43.612 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.286272803999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.324935805277544,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:43.613 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.324935805277544 }, "state": "open", "uptime": 34.286272804 } ] 2023-11-24T06:26:43.613 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:46.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:46.569 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:48.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:48.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.744+0000 7fa4a53bb700 1 Processor -- start 2023-11-24T06:26:48.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.744+0000 7fa4a53bb700 1 -- start start 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa4a53bb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa4a015e0f0 0x7fa4a00b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa4a53bb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa4a00cf2a0 0x7fa4a00d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa4a53bb700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a006bc00 con 0x7fa4a00b2b60 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa4a53bb700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a015e4d0 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa4a53bb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a006aff0 con 0x7fa4a015e0f0 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa4a00cf2a0 0x7fa4a00d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49e7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa4a00b2b60 0x7fa4a00ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57134/0 (socket says 172.21.15.114:57134) 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49e7fc700 1 -- 172.21.15.114:0/430213348 learned_addr learned my addr 172.21.15.114:0/430213348 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49f7fe700 1 -- 172.21.15.114:0/430213348 >> v2:172.21.15.172:3300/0 conn(0x7fa4a015e0f0 msgr2=0x7fa4a00b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49effd700 1 --2- 172.21.15.114:0/430213348 >> v2:172.21.15.172:3300/0 conn(0x7fa4a015e0f0 0x7fa4a00b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49f7fe700 1 --2- 172.21.15.114:0/430213348 >> v2:172.21.15.172:3300/0 conn(0x7fa4a015e0f0 0x7fa4a00b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49f7fe700 1 -- 172.21.15.114:0/430213348 >> v1:172.21.15.114:6789/0 conn(0x7fa4a00b2b60 legacy=0x7fa4a00ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49f7fe700 1 -- 172.21.15.114:0/430213348 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a006b2d0 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.745+0000 7fa49dffb700 1 -- 172.21.15.114:0/430213348 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365902509 0 0) 0x7fa4a006bc00 con 0x7fa4a00b2b60 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa49f7fe700 1 --2- 172.21.15.114:0/430213348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00cf2a0 0x7fa4a00d1f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa4a008c270 tx=0x7fa49800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8669f64d93ad3052 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa49dffb700 1 -- 172.21.15.114:0/430213348 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800e070 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa49dffb700 1 -- 172.21.15.114:0/430213348 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49800bbd0 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa49dffb700 1 -- 172.21.15.114:0/430213348 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498012430 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 -- 172.21.15.114:0/430213348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00cf2a0 msgr2=0x7fa4a00d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/430213348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00cf2a0 0x7fa4a00d1f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa4a008c270 tx=0x7fa49800b330 comp rx=0 tx=0).stop 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 -- 172.21.15.114:0/430213348 shutdown_connections 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/430213348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00cf2a0 0x7fa4a00d1f60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/430213348 >> v2:172.21.15.172:3300/0 conn(0x7fa4a015e0f0 0x7fa4a00b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 -- 172.21.15.114:0/430213348 >> 172.21.15.114:0/430213348 conn(0x7fa4a00b5ce0 msgr2=0x7fa4a00b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 -- 172.21.15.114:0/430213348 shutdown_connections 2023-11-24T06:26:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.746+0000 7fa4a53bb700 1 -- 172.21.15.114:0/430213348 wait complete. 2023-11-24T06:26:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.748+0000 7fa4a53bb700 1 Processor -- start 2023-11-24T06:26:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.748+0000 7fa4a53bb700 1 -- start start 2023-11-24T06:26:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4a00cf2a0 0x7fa4a014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4a015e0f0 0x7fa4a01f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00c8b70 con 0x7fa4a00b2b60 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00c8e10 con 0x7fa4a015e0f0 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00c90f0 con 0x7fa4a00cf2a0 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41430/0 (socket says 172.21.15.114:41430) 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 -- 172.21.15.114:0/1920949939 learned_addr learned my addr 172.21.15.114:0/1920949939 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49f7fe700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4a015e0f0 0x7fa4a01f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49e7fc700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4a00cf2a0 0x7fa4a014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 -- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4a00cf2a0 msgr2=0x7fa4a014e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4a00cf2a0 0x7fa4a014e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.756 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 -- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4a015e0f0 msgr2=0x7fa4a01f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4a015e0f0 0x7fa4a01f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 -- 172.21.15.114:0/1920949939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498009a70 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa49effd700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c7f0 tx=0x7fa49400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011800 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494011960 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a00c93d0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011cd0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.749+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01f42b0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.750+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a00751e0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.751+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa494004020 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.751+0000 7fa4877fe700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 0x7fa48c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.751+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4940895d0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.751+0000 7fa49e7fc700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 0x7fa48c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.752+0000 7fa49e7fc700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 0x7fa48c06d600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00d2bb0 tx=0x7fa490007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.756+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4940541d0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.992+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4a00683f0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.992+0000 7fa4877fe700 1 -- 172.21.15.114:0/1920949939 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fa4940577f0 con 0x7fa4a00b2b60 2023-11-24T06:26:48.996 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:48.996 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 msgr2=0x7fa48c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 0x7fa48c06d600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00d2bb0 tx=0x7fa490007de0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 msgr2=0x7fa4a01588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c7f0 tx=0x7fa49400cb00 comp rx=0 tx=0).stop 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 shutdown_connections 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa48c06b140 0x7fa48c06d600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4a015e0f0 0x7fa4a01f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4a00b2b60 0x7fa4a01588b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 --2- 172.21.15.114:0/1920949939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4a00cf2a0 0x7fa4a014e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 >> 172.21.15.114:0/1920949939 conn(0x7fa4a00b5ce0 msgr2=0x7fa4a00b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 shutdown_connections 2023-11-24T06:26:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:48.995+0000 7fa4a53bb700 1 -- 172.21.15.114:0/1920949939 wait complete. 2023-11-24T06:26:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:49.010 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:49.151 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.824155402999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.066530367728884,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:49.151 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.066530367728884 }, "state": "open", "uptime": 39.824155403 } ] 2023-11-24T06:26:49.151 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:54.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.282+0000 7f0738824700 1 Processor -- start 2023-11-24T06:26:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.282+0000 7f0738824700 1 -- start start 2023-11-24T06:26:54.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.283+0000 7f0738824700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07341473a0 0x7f073415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.283+0000 7f0738824700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0734073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.283+0000 7f0738824700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073415eb00 con 0x7f0734147780 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.283+0000 7f0738824700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073415ede0 con 0x7f07341473a0 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.283+0000 7f0738824700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073415f0c0 con 0x7f073406dc20 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f0731d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0734147780 0x7f073406c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57148/0 (socket says 172.21.15.114:57148) 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f0731d9b700 1 -- 172.21.15.114:0/1171837849 learned_addr learned my addr 172.21.15.114:0/1171837849 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f0732d9d700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0734073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073259c700 1 --2- 172.21.15.114:0/1171837849 >> v2:172.21.15.172:3300/0 conn(0x7f07341473a0 0x7f073415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668786133 0 0) 0x7f073415eb00 con 0x7f0734147780 2023-11-24T06:26:54.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07200035b0 con 0x7f0734147780 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917674725 0 0) 0x7f07200035b0 con 0x7f0734147780 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 msgr2=0x7f0734073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0734073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 >> v2:172.21.15.172:3300/0 conn(0x7f07341473a0 msgr2=0x7f073415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 --2- 172.21.15.114:0/1171837849 >> v2:172.21.15.172:3300/0 conn(0x7f07341473a0 0x7f073415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073415f3a0 con 0x7f0734147780 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f071c001670 con 0x7f0734147780 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 >> v1:172.21.15.114:6789/0 conn(0x7f0734147780 legacy=0x7f073406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0720003a00 0x7f0720003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0720004850 0x7f0720004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0720005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073415ede0 con 0x7f0720004850 2023-11-24T06:26:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073415f0c0 con 0x7f0720003a00 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072000ba10 con 0x7f073406dc20 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f0731d9b700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0720004850 0x7f0720004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.284+0000 7f073259c700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0720003a00 0x7f0720003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0732d9d700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0720005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 -- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 msgr2=0x7f0720005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073406dc20 0x7f0720005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 -- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0720003a00 msgr2=0x7f0720003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0720003a00 0x7f0720003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 -- 172.21.15.114:0/1171837849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07200035b0 con 0x7f0720004850 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0731d9b700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0720004850 0x7f0720004cd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0734147c00 tx=0x7f0720005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5182e7c80c73b205 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00ce00 con 0x7f0720004850 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c005960 con 0x7f0720004850 2023-11-24T06:26:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f073159a700 1 -- 172.21.15.114:0/1171837849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c005d10 con 0x7f0720004850 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0738824700 1 -- 172.21.15.114:0/1171837849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0720004850 msgr2=0x7f0720004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0738824700 1 --2- 172.21.15.114:0/1171837849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0720004850 0x7f0720004cd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0734147c00 tx=0x7f0720005880 comp rx=0 tx=0).stop 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.285+0000 7f0732d9d700 1 -- 172.21.15.114:0/1171837849 reap_dead start 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.286+0000 7f0738824700 1 -- 172.21.15.114:0/1171837849 shutdown_connections 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.286+0000 7f0738824700 1 -- 172.21.15.114:0/1171837849 >> 172.21.15.114:0/1171837849 conn(0x7f07340b3130 msgr2=0x7f07340d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.286+0000 7f0738824700 1 -- 172.21.15.114:0/1171837849 shutdown_connections 2023-11-24T06:26:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.286+0000 7f0738824700 1 -- 172.21.15.114:0/1171837849 wait complete. 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.287+0000 7f0738824700 1 Processor -- start 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.287+0000 7f0738824700 1 -- start start 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0720004850 0x7f0734160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073413e8d0 0x7f07340cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734161450 con 0x7f07340af7e0 2023-11-24T06:26:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340cdab0 con 0x7f073413e8d0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340aea20 con 0x7f0720004850 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41454/0 (socket says 172.21.15.114:41454) 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 -- 172.21.15.114:0/4172064783 learned_addr learned my addr 172.21.15.114:0/4172064783 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f073259c700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0720004850 0x7f0734160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0731d9b700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073413e8d0 0x7f07340cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 -- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0720004850 msgr2=0x7f0734160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0720004850 0x7f0734160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 -- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073413e8d0 msgr2=0x7f07340cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073413e8d0 0x7f07340cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 -- 172.21.15.114:0/4172064783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340aed00 con 0x7f07340af7e0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0732d9d700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f07340b3830 tx=0x7f072c011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c004230 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c004390 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07340aefe0 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.288+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c0036a0 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.289+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07340afc90 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.289+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0734067f60 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.290+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f072c003800 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.290+0000 7f072affd700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 0x7f071406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.291+0000 7f073259c700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 0x7f071406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.291+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f072c089a70 con 0x7f07340af7e0 2023-11-24T06:26:54.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.291+0000 7f073259c700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 0x7f071406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0724014450 tx=0x7f072400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.295+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f072c054810 con 0x7f07340af7e0 2023-11-24T06:26:54.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.525+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f073415f680 con 0x7f07340af7e0 2023-11-24T06:26:54.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.525+0000 7f072affd700 1 -- 172.21.15.114:0/4172064783 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f072c057e30 con 0x7f07340af7e0 2023-11-24T06:26:54.531 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:26:54.531 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:26:54.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 msgr2=0x7f071406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 0x7f071406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0724014450 tx=0x7f072400b040 comp rx=0 tx=0).stop 2023-11-24T06:26:54.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 msgr2=0x7f07341433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f07340b3830 tx=0x7f072c011770 comp rx=0 tx=0).stop 2023-11-24T06:26:54.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 shutdown_connections 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f071406b110 0x7f071406d5d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07340af7e0 0x7f07341433e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0720004850 0x7f0734160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 --2- 172.21.15.114:0/4172064783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073413e8d0 0x7f07340cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 >> 172.21.15.114:0/4172064783 conn(0x7f07340b3130 msgr2=0x7f073406b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 shutdown_connections 2023-11-24T06:26:54.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:54.527+0000 7f0738824700 1 -- 172.21.15.114:0/4172064783 wait complete. 2023-11-24T06:26:54.534 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:26:54.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:26:54.683 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.356038130000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.811684863728289,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:26:54.684 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.81168486372829 }, "state": "open", "uptime": 45.35603813 } ] 2023-11-24T06:26:54.684 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:26:59.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:59.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.818+0000 7f4ab2e00700 1 Processor -- start 2023-11-24T06:26:59.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.818+0000 7f4ab2e00700 1 -- start start 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4ab2e00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4aac0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4ab2e00700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0af8c0 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4ab2e00700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0b1030 con 0x7f4aac15afc0 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4ab2e00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0b0790 con 0x7f4aac0b1b10 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4aabfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4aac0b0bb0 0x7f4aac0b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57168/0 (socket says 172.21.15.114:57168) 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4aabfff700 1 -- 172.21.15.114:0/354584948 learned_addr learned my addr 172.21.15.114:0/354584948 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765852984 0 0) 0x7f4aac0af8c0 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4ab139d700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4aac0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.819+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a9c0035b0 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4152101511 0 0) 0x7f4a9c0035b0 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 msgr2=0x7f4aac0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4aac0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 >> v1:172.21.15.172:6789/0 conn(0x7f4aac15afc0 legacy=0x7f4aac0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0afb10 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f4a98001670 con 0x7f4aac0b0bb0 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 >> v1:172.21.15.114:6789/0 conn(0x7f4aac0b0bb0 legacy=0x7f4aac0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4a9c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a9c003fb0 0x7f4a9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c004d90 0x7f4a9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0b0790 con 0x7f4a9c003fb0 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c00baf0 con 0x7f4a9c004d90 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c00bcb0 con 0x7f4aac0b1b10 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a9c003fb0 0x7f4a9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4ab0b9c700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4a9c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4ab139d700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c004d90 0x7f4a9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 -- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 msgr2=0x7f4a9c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b1b10 0x7f4a9c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 -- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c004d90 msgr2=0x7f4a9c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c004d90 0x7f4a9c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 -- 172.21.15.114:0/354584948 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0035b0 con 0x7f4a9c003fb0 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4ab139d700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c004d90 0x7f4a9c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aabfff700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a9c003fb0 0x7f4a9c004850 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c2a0 tx=0x7f4a9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b200a35de07ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98007c10 con 0x7f4a9c003fb0 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9800cdc0 con 0x7f4a9c003fb0 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.820+0000 7f4aab7fe700 1 -- 172.21.15.114:0/354584948 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98005ba0 con 0x7f4a9c003fb0 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 -- 172.21.15.114:0/354584948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a9c003fb0 msgr2=0x7f4a9c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/354584948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a9c003fb0 0x7f4a9c004850 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c2a0 tx=0x7f4a9c005880 comp rx=0 tx=0).stop 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab139d700 1 -- 172.21.15.114:0/354584948 reap_dead start 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 -- 172.21.15.114:0/354584948 shutdown_connections 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 -- 172.21.15.114:0/354584948 >> 172.21.15.114:0/354584948 conn(0x7f4aac0b57b0 msgr2=0x7f4aac0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 -- 172.21.15.114:0/354584948 shutdown_connections 2023-11-24T06:26:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.821+0000 7f4ab2e00700 1 -- 172.21.15.114:0/354584948 wait complete. 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.822+0000 7f4ab2e00700 1 Processor -- start 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.822+0000 7f4ab2e00700 1 -- start start 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c003fb0 0x7f4aac0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 0x7f4aac15d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cbd80 con 0x7f4aac14e310 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cb170 con 0x7f4a9c003fb0 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab2e00700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cb450 con 0x7f4aac0cac20 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.823+0000 7f4ab139d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41474/0 (socket says 172.21.15.114:41474) 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 -- 172.21.15.114:0/2268129108 learned_addr learned my addr 172.21.15.114:0/2268129108 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab0b9c700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c003fb0 0x7f4aac0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4aabfff700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 0x7f4aac15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 -- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 msgr2=0x7f4aac15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 0x7f4aac15d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 -- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c003fb0 msgr2=0x7f4aac0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c003fb0 0x7f4aac0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 -- 172.21.15.114:0/2268129108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0cb730 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4aabfff700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 0x7f4aac15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab139d700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0b5760 tx=0x7f4aa40104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa401b070 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa400f040 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac0cba10 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400e050 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.824+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac14ece0 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.825+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac0cbef0 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.826+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4aa4003a20 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.826+0000 7f4aa97fa700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 0x7f4a9006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.826+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4aa4089e10 con 0x7f4aac14e310 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.826+0000 7f4ab0b9c700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 0x7f4a9006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.826+0000 7f4ab0b9c700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 0x7f4a9006d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ad770 tx=0x7f4aa0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:26:59.831+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4aa4054a10 con 0x7f4aac14e310 2023-11-24T06:27:00.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.062+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aac067f60 con 0x7f4aac14e310 2023-11-24T06:27:00.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.062+0000 7f4aa97fa700 1 -- 172.21.15.114:0/2268129108 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f4aa4058030 con 0x7f4aac14e310 2023-11-24T06:27:00.066 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:00.066 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:00.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.064+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 msgr2=0x7f4a9006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.064+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 0x7f4a9006d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ad770 tx=0x7f4aa0009040 comp rx=0 tx=0).stop 2023-11-24T06:27:00.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.064+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 msgr2=0x7f4aac14e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.064+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0b5760 tx=0x7f4aa40104a0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 shutdown_connections 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4a9006b110 0x7f4a9006d5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4aac14e310 0x7f4aac14e7a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a9c003fb0 0x7f4aac0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 --2- 172.21.15.114:0/2268129108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0cac20 0x7f4aac15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 >> 172.21.15.114:0/2268129108 conn(0x7f4aac0b57b0 msgr2=0x7f4aac0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 shutdown_connections 2023-11-24T06:27:00.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:00.065+0000 7f4ab2e00700 1 -- 172.21.15.114:0/2268129108 wait complete. 2023-11-24T06:27:00.070 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:00.082 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:00.222 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.895920689999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.559714368762446,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:27:00.223 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.559714368762446 }, "state": "open", "uptime": 50.89592069 } ] 2023-11-24T06:27:00.223 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:27:05.224 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.353+0000 7efc038df700 1 Processor -- start 2023-11-24T06:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.354+0000 7efc038df700 1 -- start start 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc038df700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc15bef0 0x7efbfc15e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc038df700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efbfc1624d0 0x7efbfc162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc038df700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc06dd80 con 0x7efbfc0afe70 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc038df700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc15f880 con 0x7efbfc1624d0 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc038df700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc15ed90 con 0x7efbfc15bef0 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efbfc1624d0 0x7efbfc162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7efbfc1624d0 0x7efbfc162960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41482/0 (socket says 172.21.15.114:41482) 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 -- 172.21.15.114:0/1503335971 learned_addr learned my addr 172.21.15.114:0/1503335971 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 -- 172.21.15.114:0/1503335971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc15bef0 msgr2=0x7efbfc15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 --2- 172.21.15.114:0/1503335971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc15bef0 0x7efbfc15e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 -- 172.21.15.114:0/1503335971 >> v1:172.21.15.172:6789/0 conn(0x7efbfc0afe70 legacy=0x7efbfc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 -- 172.21.15.114:0/1503335971 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc15c2d0 con 0x7efbfc1624d0 2023-11-24T06:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efc01e7c700 1 --2- 172.21.15.114:0/1503335971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc1624d0 0x7efbfc162960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009e50 tx=0x7efbf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd2010ba93fe0f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efbeffff700 1 -- 172.21.15.114:0/1503335971 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400e070 con 0x7efbfc1624d0 2023-11-24T06:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efbeffff700 1 -- 172.21.15.114:0/1503335971 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf4003a20 con 0x7efbfc1624d0 2023-11-24T06:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.355+0000 7efbeffff700 1 -- 172.21.15.114:0/1503335971 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003dd0 con 0x7efbfc1624d0 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 -- 172.21.15.114:0/1503335971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc1624d0 msgr2=0x7efbfc162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 --2- 172.21.15.114:0/1503335971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc1624d0 0x7efbfc162960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009e50 tx=0x7efbf400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 -- 172.21.15.114:0/1503335971 shutdown_connections 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 --2- 172.21.15.114:0/1503335971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc1624d0 0x7efbfc162960 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 --2- 172.21.15.114:0/1503335971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc15bef0 0x7efbfc15e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 -- 172.21.15.114:0/1503335971 >> 172.21.15.114:0/1503335971 conn(0x7efbfc071ac0 msgr2=0x7efbfc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 -- 172.21.15.114:0/1503335971 shutdown_connections 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.356+0000 7efc038df700 1 -- 172.21.15.114:0/1503335971 wait complete. 2023-11-24T06:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.358+0000 7efc038df700 1 Processor -- start 2023-11-24T06:27:05.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.358+0000 7efc038df700 1 -- start start 2023-11-24T06:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 0x7efbfc15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 0x7efbfc148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc15d2d0 con 0x7efbfc15bef0 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc15d5b0 con 0x7efbfc1624d0 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc15d890 con 0x7efbfc0afe70 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41486/0 (socket says 172.21.15.114:41486) 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 -- 172.21.15.114:0/2531421311 learned_addr learned my addr 172.21.15.114:0/2531421311 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc0167b700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 0x7efbfc15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc01e7c700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 0x7efbfc148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 -- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 msgr2=0x7efbfc15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:05.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 0x7efbfc15ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 -- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 msgr2=0x7efbfc148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 0x7efbfc148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 -- 172.21.15.114:0/2531421311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4009b30 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc0167b700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 0x7efbfc15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc01e7c700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 0x7efbfc148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc00e7a700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b470 tx=0x7efbf800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8011830 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf8011990 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc15db70 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.359+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8011d40 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.360+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc15cfc0 con 0x7efbfc15bef0 2023-11-24T06:27:05.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.360+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc067f60 con 0x7efbfc15bef0 2023-11-24T06:27:05.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.361+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7efbf801a400 con 0x7efbfc15bef0 2023-11-24T06:27:05.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.361+0000 7efbedffb700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 0x7efbe006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:05.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.361+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efbf8089af0 con 0x7efbfc15bef0 2023-11-24T06:27:05.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.361+0000 7efc0167b700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 0x7efbe006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:05.367 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.362+0000 7efc0167b700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 0x7efbe006d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efbf00099d0 tx=0x7efbf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:05.369 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.366+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efbf8054800 con 0x7efbfc15bef0 2023-11-24T06:27:05.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.597+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efbfc0b41b0 con 0x7efbfc15bef0 2023-11-24T06:27:05.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.598+0000 7efbedffb700 1 -- 172.21.15.114:0/2531421311 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7efbf8015020 con 0x7efbfc15bef0 2023-11-24T06:27:05.602 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:05.602 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 msgr2=0x7efbe006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 0x7efbe006d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efbf00099d0 tx=0x7efbf0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 msgr2=0x7efbfc0aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b470 tx=0x7efbf800b830 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 shutdown_connections 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7efbe006b110 0x7efbe006d5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbfc1624d0 0x7efbfc148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbfc0afe70 0x7efbfc15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 --2- 172.21.15.114:0/2531421311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efbfc15bef0 0x7efbfc0aec60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 >> 172.21.15.114:0/2531421311 conn(0x7efbfc071ac0 msgr2=0x7efbfc0b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:05.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 shutdown_connections 2023-11-24T06:27:05.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:05.600+0000 7efc038df700 1 -- 172.21.15.114:0/2531421311 wait complete. 2023-11-24T06:27:05.605 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:05.615 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:05.756 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.428803395000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.311216207248226,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:27:05.757 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.311216207248226 }, "state": "open", "uptime": 56.428803395 } ] 2023-11-24T06:27:05.757 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:27:10.758 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:10.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.889+0000 7f0182b37700 1 Processor -- start 2023-11-24T06:27:10.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.889+0000 7f0182b37700 1 -- start start 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f0182b37700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f017c15e080 0x7f017c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f0182b37700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f017c06c710 0x7f017c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f0182b37700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017c0ae430 con 0x7f017c0af4d0 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f0182b37700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c0af0b0 con 0x7f017c06c710 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f0182b37700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c0b05a0 con 0x7f017c15e080 2023-11-24T06:27:10.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f017c06c710 0x7f017c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f017c06c710 0x7f017c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41494/0 (socket says 172.21.15.114:41494) 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 -- 172.21.15.114:0/1149085602 learned_addr learned my addr 172.21.15.114:0/1149085602 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 -- 172.21.15.114:0/1149085602 >> v2:172.21.15.172:3300/0 conn(0x7f017c15e080 msgr2=0x7f017c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 --2- 172.21.15.114:0/1149085602 >> v2:172.21.15.172:3300/0 conn(0x7f017c15e080 0x7f017c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 -- 172.21.15.114:0/1149085602 >> v1:172.21.15.172:6789/0 conn(0x7f017c0af4d0 legacy=0x7f017c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 -- 172.21.15.114:0/1149085602 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0ae710 con 0x7f017c06c710 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.890+0000 7f01810d4700 1 --2- 172.21.15.114:0/1149085602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c06c710 0x7f017c0afb90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f017c072a90 tx=0x7f017400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afefc8ca8f126bb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f017b7fe700 1 -- 172.21.15.114:0/1149085602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400e070 con 0x7f017c06c710 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f017b7fe700 1 -- 172.21.15.114:0/1149085602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0174003a20 con 0x7f017c06c710 2023-11-24T06:27:10.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f017b7fe700 1 -- 172.21.15.114:0/1149085602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174003dd0 con 0x7f017c06c710 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 -- 172.21.15.114:0/1149085602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c06c710 msgr2=0x7f017c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 --2- 172.21.15.114:0/1149085602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c06c710 0x7f017c0afb90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f017c072a90 tx=0x7f017400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 -- 172.21.15.114:0/1149085602 shutdown_connections 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 --2- 172.21.15.114:0/1149085602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c06c710 0x7f017c0afb90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 --2- 172.21.15.114:0/1149085602 >> v2:172.21.15.172:3300/0 conn(0x7f017c15e080 0x7f017c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 -- 172.21.15.114:0/1149085602 >> 172.21.15.114:0/1149085602 conn(0x7f017c071b90 msgr2=0x7f017c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:10.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 -- 172.21.15.114:0/1149085602 shutdown_connections 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.891+0000 7f0182b37700 1 -- 172.21.15.114:0/1149085602 wait complete. 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.892+0000 7f0182b37700 1 Processor -- start 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.892+0000 7f0182b37700 1 -- start start 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 0x7f017c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c15e080 0x7f017c159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c1553d0 con 0x7f017c0af4d0 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c15a360 con 0x7f017c15e080 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f0182b37700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017c157df0 con 0x7f017c06c710 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f017bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f017bfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41498/0 (socket says 172.21.15.114:41498) 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.893+0000 7f017bfff700 1 -- 172.21.15.114:0/3593246076 learned_addr learned my addr 172.21.15.114:0/3593246076 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01810d4700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c15e080 0x7f017c159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01808d3700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 0x7f017c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 -- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 msgr2=0x7f017c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 0x7f017c153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 -- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c15e080 msgr2=0x7f017c159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c15e080 0x7f017c159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 -- 172.21.15.114:0/3593246076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174009b30 con 0x7f017c0af4d0 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01808d3700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 0x7f017c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f017bfff700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f016800c4a0 tx=0x7f016800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.899 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680054e0 con 0x7f017c0af4d0 2023-11-24T06:27:10.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168003680 con 0x7f017c0af4d0 2023-11-24T06:27:10.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168003a10 con 0x7f017c0af4d0 2023-11-24T06:27:10.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c1580d0 con 0x7f017c0af4d0 2023-11-24T06:27:10.900 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.894+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c158690 con 0x7f017c0af4d0 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.895+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c0753f0 con 0x7f017c0af4d0 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.895+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0168003b70 con 0x7f017c0af4d0 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.896+0000 7f01797fa700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 0x7f016006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.896+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0168088bb0 con 0x7f017c0af4d0 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.896+0000 7f01808d3700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 0x7f016006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.896+0000 7f01808d3700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 0x7f016006d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f017c160f90 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:10.900+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0168053950 con 0x7f017c0af4d0 2023-11-24T06:27:11.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.138+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f017c0b2780 con 0x7f017c0af4d0 2023-11-24T06:27:11.141 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.139+0000 7f01797fa700 1 -- 172.21.15.114:0/3593246076 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f0168017090 con 0x7f017c0af4d0 2023-11-24T06:27:11.141 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:11.142 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:11.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 msgr2=0x7f016006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 0x7f016006d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f017c160f90 tx=0x7f0170008040 comp rx=0 tx=0).stop 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 msgr2=0x7f017c154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f016800c4a0 tx=0x7f016800c860 comp rx=0 tx=0).stop 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 shutdown_connections 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f016006b110 0x7f016006d5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f017c15e080 0x7f017c159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f017c06c710 0x7f017c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 --2- 172.21.15.114:0/3593246076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f017c0af4d0 0x7f017c154e90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 >> 172.21.15.114:0/3593246076 conn(0x7f017c071b90 msgr2=0x7f017c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 shutdown_connections 2023-11-24T06:27:11.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:11.141+0000 7f0182b37700 1 -- 172.21.15.114:0/3593246076 wait complete. 2023-11-24T06:27:11.145 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:11.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:11.295 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.968685952000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.06565486143344,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:27:11.295 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.06565486143344 }, "state": "open", "uptime": 61.968685952 } ] 2023-11-24T06:27:11.295 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:27:16.296 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.427+0000 7f7502fa8700 1 Processor -- start 2023-11-24T06:27:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.427+0000 7f7502fa8700 1 -- start start 2023-11-24T06:27:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7502fa8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc15bdd0 0x7f74fc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7502fa8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc0b3370 0x7f74fc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7502fa8700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc0b2030 con 0x7f74fc0d1c70 2023-11-24T06:27:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7502fa8700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74fc0b2f50 con 0x7f74fc15bdd0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7502fa8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0b22d0 con 0x7f74fc0b3370 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7500d44700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc15bdd0 0x7f74fc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7500d44700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc15bdd0 0x7f74fc1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41504/0 (socket says 172.21.15.114:41504) 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f7500d44700 1 -- 172.21.15.114:0/579133693 learned_addr learned my addr 172.21.15.114:0/579133693 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.428+0000 7f74fbfff700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc0b3370 0x7f74fc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7500d44700 1 -- 172.21.15.114:0/579133693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc0b3370 msgr2=0x7f74fc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7500d44700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc0b3370 0x7f74fc0b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7500d44700 1 -- 172.21.15.114:0/579133693 >> v1:172.21.15.172:6789/0 conn(0x7f74fc0d1c70 legacy=0x7f74fc0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7500d44700 1 -- 172.21.15.114:0/579133693 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0b25b0 con 0x7f74fc15bdd0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7500d44700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc15bdd0 0x7f74fc1604e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f74f00098f0 tx=0x7f74f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadb171000249712 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f74fb7fe700 1 -- 172.21.15.114:0/579133693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f000b8a0 con 0x7f74fc15bdd0 2023-11-24T06:27:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f74fb7fe700 1 -- 172.21.15.114:0/579133693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f000ba00 con 0x7f74fc15bdd0 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f74fb7fe700 1 -- 172.21.15.114:0/579133693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f000bd90 con 0x7f74fc15bdd0 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 -- 172.21.15.114:0/579133693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc15bdd0 msgr2=0x7f74fc1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc15bdd0 0x7f74fc1604e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f74f00098f0 tx=0x7f74f0009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 -- 172.21.15.114:0/579133693 shutdown_connections 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc15bdd0 0x7f74fc1604e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 --2- 172.21.15.114:0/579133693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc0b3370 0x7f74fc0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.429+0000 7f7502fa8700 1 -- 172.21.15.114:0/579133693 >> 172.21.15.114:0/579133693 conn(0x7f74fc0731a0 msgr2=0x7f74fc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.430+0000 7f7502fa8700 1 -- 172.21.15.114:0/579133693 shutdown_connections 2023-11-24T06:27:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.430+0000 7f7502fa8700 1 -- 172.21.15.114:0/579133693 wait complete. 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.431+0000 7f7502fa8700 1 Processor -- start 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.431+0000 7f7502fa8700 1 -- start start 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74fc0b3370 0x7f74fc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc15bdd0 0x7f74fc0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0b48d0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0c9920 con 0x7f74fc0b3370 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0c9c00 con 0x7f74fc15bdd0 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41512/0 (socket says 172.21.15.114:41512) 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 -- 172.21.15.114:0/2393993199 learned_addr learned my addr 172.21.15.114:0/2393993199 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7500d44700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74fc0b3370 0x7f74fc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7501545700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc15bdd0 0x7f74fc0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 -- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc15bdd0 msgr2=0x7f74fc0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc15bdd0 0x7f74fc0c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 -- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74fc0b3370 msgr2=0x7f74fc0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74fc0b3370 0x7f74fc0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 -- 172.21.15.114:0/2393993199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0009550 con 0x7f74fc0d1c70 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74fbfff700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74e800e810 tx=0x7f74e800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800c070 con 0x7f74fc0d1c70 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e8013610 con 0x7f74fc0d1c70 2023-11-24T06:27:16.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc0c9ee0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.432+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80139c0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.433+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc0ca560 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.433+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc0ca720 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.434+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f74e8013b20 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.434+0000 7f74f97fa700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 0x7f74e006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.434+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f74e80898b0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.434+0000 7f7500d44700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 0x7f74e006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:16.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.435+0000 7f7500d44700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 0x7f74e006d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74f0014040 tx=0x7f74f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:16.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.439+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74e80544c0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.638 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:16.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.677+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74fc0af190 con 0x7f74fc0d1c70 2023-11-24T06:27:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.677+0000 7f74f97fa700 1 -- 172.21.15.114:0/2393993199 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f74e8057ae0 con 0x7f74fc0d1c70 2023-11-24T06:27:16.680 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:16.680 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 msgr2=0x7f74e006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 0x7f74e006d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74f0014040 tx=0x7f74f0008030 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 msgr2=0x7f74fc0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f74e800e810 tx=0x7f74e800eb20 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 shutdown_connections 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f74e006b110 0x7f74e006d5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74fc15bdd0 0x7f74fc0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74fc0b3370 0x7f74fc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 --2- 172.21.15.114:0/2393993199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0d1c70 0x7f74fc0c8db0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 >> 172.21.15.114:0/2393993199 conn(0x7f74fc0731a0 msgr2=0x7f74fc0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 shutdown_connections 2023-11-24T06:27:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:16.679+0000 7f7502fa8700 1 -- 172.21.15.114:0/2393993199 wait complete. 2023-11-24T06:27:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:16.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:16.875 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.548567657999996,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.82143306963988,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:27:16.875 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.82143306963988 }, "state": "open", "uptime": 67.548567658 } ] 2023-11-24T06:27:16.876 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:27:21.877 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:22.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.007+0000 7f57ab8fe700 1 Processor -- start 2023-11-24T06:27:22.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.007+0000 7f57ab8fe700 1 -- start start 2023-11-24T06:27:22.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57ab8fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f57a40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57ab8fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f57a406c710 0x7f57a40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57ab8fe700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40ae430 con 0x7f57a40af4d0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57ab8fe700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a40af0b0 con 0x7f57a406c710 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57ab8fe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40b05a0 con 0x7f57a415e080 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57a8e99700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f57a40af4d0 0x7f57a406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57230/0 (socket says 172.21.15.114:57230) 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.008+0000 7f57a8e99700 1 -- 172.21.15.114:0/3106742242 learned_addr learned my addr 172.21.15.114:0/3106742242 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a9e9b700 1 --2- 172.21.15.114:0/3106742242 >> v2:172.21.15.172:3300/0 conn(0x7f57a406c710 0x7f57a40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154381803 0 0) 0x7f57a40ae430 con 0x7f57a40af4d0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57900035b0 con 0x7f57a40af4d0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f57a40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340031206 0 0) 0x7f57900035b0 con 0x7f57a40af4d0 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 msgr2=0x7f57a40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f57a40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 >> v2:172.21.15.172:3300/0 conn(0x7f57a406c710 msgr2=0x7f57a40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 --2- 172.21.15.114:0/3106742242 >> v2:172.21.15.172:3300/0 conn(0x7f57a406c710 0x7f57a40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40ae710 con 0x7f57a40af4d0 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f57a0001670 con 0x7f57a40af4d0 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 >> v1:172.21.15.114:6789/0 conn(0x7f57a40af4d0 legacy=0x7f57a406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5790003a30 0x7f5790003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f5790004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5790004d90 0x7f5790005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40af0b0 con 0x7f5790003a30 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a40b05a0 con 0x7f5790004d90 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579000ba10 con 0x7f57a415e080 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5790003a30 0x7f5790003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a9e9b700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5790004d90 0x7f5790005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a8e99700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f5790004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 -- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 msgr2=0x7f5790004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f57a415e080 0x7f5790004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 -- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5790004d90 msgr2=0x7f5790005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5790004d90 0x7f5790005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.009+0000 7f57a969a700 1 -- 172.21.15.114:0/3106742242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900035b0 con 0x7f5790003a30 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57a969a700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5790003a30 0x7f5790003e50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5794009b90 tx=0x7f5790005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad32faa32a8bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400e6c0 con 0x7f5790003a30 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579400e820 con 0x7f5790003a30 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f579bfff700 1 -- 172.21.15.114:0/3106742242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400eb90 con 0x7f5790003a30 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 -- 172.21.15.114:0/3106742242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5790003a30 msgr2=0x7f5790003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/3106742242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5790003a30 0x7f5790003e50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5794009b90 tx=0x7f5790005880 comp rx=0 tx=0).stop 2023-11-24T06:27:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57a9e9b700 1 -- 172.21.15.114:0/3106742242 reap_dead start 2023-11-24T06:27:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 -- 172.21.15.114:0/3106742242 shutdown_connections 2023-11-24T06:27:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 -- 172.21.15.114:0/3106742242 >> 172.21.15.114:0/3106742242 conn(0x7f57a4071b90 msgr2=0x7f57a4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 -- 172.21.15.114:0/3106742242 shutdown_connections 2023-11-24T06:27:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.010+0000 7f57ab8fe700 1 -- 172.21.15.114:0/3106742242 wait complete. 2023-11-24T06:27:22.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 Processor -- start 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 -- start start 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5790003a30 0x7f57a4149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a4156070 0x7f57a41f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4156520 con 0x7f57a415a5e0 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4149e70 con 0x7f57a4156070 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57ab8fe700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41542b0 con 0x7f5790003a30 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57a8e99700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.012+0000 7f57a8e99700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41532/0 (socket says 172.21.15.114:41532) 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 -- 172.21.15.114:0/772860015 learned_addr learned my addr 172.21.15.114:0/772860015 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a969a700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5790003a30 0x7f57a4149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a9e9b700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a4156070 0x7f57a41f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 -- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5790003a30 msgr2=0x7f57a4149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5790003a30 0x7f57a4149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 -- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a4156070 msgr2=0x7f57a41f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a4156070 0x7f57a41f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 -- 172.21.15.114:0/772860015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4154550 con 0x7f57a415a5e0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57a8e99700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e810 tx=0x7f57a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0012590 con 0x7f57a415a5e0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0012710 con 0x7f57a415a5e0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0012ac0 con 0x7f57a415a5e0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a4154830 con 0x7f57a415a5e0 2023-11-24T06:27:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.013+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a4154df0 con 0x7f57a415a5e0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.014+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a40753f0 con 0x7f57a415a5e0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.015+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f57a0012c20 con 0x7f57a415a5e0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.015+0000 7f5799ffb700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 0x7f578806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.015+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f57a0089710 con 0x7f57a415a5e0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.015+0000 7f57a969a700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 0x7f578806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.016+0000 7f57a969a700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 0x7f578806d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f57940090b0 tx=0x7f579401f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:22.022 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.020+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57a0057f80 con 0x7f57a415a5e0 2023-11-24T06:27:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.252+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57a4067f60 con 0x7f57a415a5e0 2023-11-24T06:27:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.253+0000 7f5799ffb700 1 -- 172.21.15.114:0/772860015 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f57a0012ed0 con 0x7f57a415a5e0 2023-11-24T06:27:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:22.256 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 msgr2=0x7f578806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 0x7f578806d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f57940090b0 tx=0x7f579401f040 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 msgr2=0x7f57a4155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f57a000e810 tx=0x7f57a000eb20 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 shutdown_connections 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f578806b110 0x7f578806d5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57a4156070 0x7f57a41f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5790003a30 0x7f57a4149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 --2- 172.21.15.114:0/772860015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57a415a5e0 0x7f57a4155b30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 >> 172.21.15.114:0/772860015 conn(0x7f57a4071b90 msgr2=0x7f57a40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 shutdown_connections 2023-11-24T06:27:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:22.255+0000 7f57ab8fe700 1 -- 172.21.15.114:0/772860015 wait complete. 2023-11-24T06:27:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:22.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:22.409 INFO:teuthology.orchestra.run.smithi114.stdout:[{"id":5952,"entity":{"name":{"type":"client","num":5952},"addr":{"type":"any","addr":"172.21.15.114:0","nonce":3472748561}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":73.082450340999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.582314469753875,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5952 172.21.15.114:0/3472748561","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"a13ea5ff00cb4df7a9cc941166ddeb358881144c","ceph_version":"ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)","entity_id":"y","hostname":"smithi114","pid":"103262","root":"/"}}] 2023-11-24T06:27:22.409 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi114", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103262", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.114:0", "nonce": 3472748561, "type": "any" }, "name": { "num": 5952, "type": "client" } }, "id": 5952, "inst": "client.5952 172.21.15.114:0/3472748561", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.582314469753875 }, "state": "open", "uptime": 73.082450341 } ] 2023-11-24T06:27:22.410 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-24T06:27:27.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.543+0000 7f40eb709700 1 Processor -- start 2023-11-24T06:27:27.546 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.543+0000 7f40eb709700 1 -- start start 2023-11-24T06:27:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40eb709700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40eb709700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40ae430 con 0x7f40e40af4d0 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40eb709700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40ae710 con 0x7f40e406c710 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40eb709700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40af0b0 con 0x7f40e415e080 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40e8ca4700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f40e40af4d0 0x7f40e406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57246/0 (socket says 172.21.15.114:57246) 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40e8ca4700 1 -- 172.21.15.114:0/2316702463 learned_addr learned my addr 172.21.15.114:0/2316702463 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40e94a5700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.544+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679305914 0 0) 0x7f40e40ae430 con 0x7f40e40af4d0 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d00035b0 con 0x7f40e40af4d0 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557103206 0 0) 0x7f40e40ae710 con 0x7f40e406c710 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e40ae430 con 0x7f40e406c710 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908477216 0 0) 0x7f40d00035b0 con 0x7f40e40af4d0 2023-11-24T06:27:27.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 msgr2=0x7f40e40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40e40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 >> v1:172.21.15.172:6789/0 conn(0x7f40e406c710 legacy=0x7f40e406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40ae9f0 con 0x7f40e40af4d0 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f40e0001670 con 0x7f40e40af4d0 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 >> v1:172.21.15.114:6789/0 conn(0x7f40e40af4d0 legacy=0x7f40e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40d0003bc0 0x7f40d0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40d0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40af0b0 con 0x7f40d0004850 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d000bac0 con 0x7f40d0003bc0 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d000bc50 con 0x7f40e415e080 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40d0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.551 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e9ca6700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e94a5700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40d0003bc0 0x7f40d0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 -- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 msgr2=0x7f40d0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40d0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 -- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40d0003bc0 msgr2=0x7f40d0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40d0003bc0 0x7f40d0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 -- 172.21.15.114:0/2316702463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00035b0 con 0x7f40d0004850 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e94a5700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40d0003bc0 0x7f40d0003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e9ca6700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e415e080 0x7f40d0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.545+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40d0004cd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f40e40af950 tx=0x7f40d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed802b8b3d8456d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:27.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cd40 con 0x7f40d0004850 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0001af0 con 0x7f40d0004850 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40dbfff700 1 -- 172.21.15.114:0/2316702463 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cea0 con 0x7f40d0004850 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 -- 172.21.15.114:0/2316702463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 msgr2=0x7f40d0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 --2- 172.21.15.114:0/2316702463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40d0004cd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f40e40af950 tx=0x7f40d0005880 comp rx=0 tx=0).stop 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40e9ca6700 1 -- 172.21.15.114:0/2316702463 reap_dead start 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 -- 172.21.15.114:0/2316702463 shutdown_connections 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 -- 172.21.15.114:0/2316702463 >> 172.21.15.114:0/2316702463 conn(0x7f40e4071b90 msgr2=0x7f40e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 -- 172.21.15.114:0/2316702463 shutdown_connections 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.546+0000 7f40eb709700 1 -- 172.21.15.114:0/2316702463 wait complete. 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.547+0000 7f40eb709700 1 Processor -- start 2023-11-24T06:27:27.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 -- start start 2023-11-24T06:27:27.555 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e4157130 0x7f40e41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 0x7f40e4159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4158310 con 0x7f40d0004850 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4157ca0 con 0x7f40e4158db0 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40eb709700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4157f80 con 0x7f40e4157130 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40e94a5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40e94a5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41550/0 (socket says 172.21.15.114:41550) 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40e94a5700 1 -- 172.21.15.114:0/3832278171 learned_addr learned my addr 172.21.15.114:0/3832278171 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.548+0000 7f40e9ca6700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 0x7f40e4159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e4157130 0x7f40e41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.556 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 -- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e4157130 msgr2=0x7f40e41575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e4157130 0x7f40e41575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 -- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 msgr2=0x7f40e4159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 0x7f40e4159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 -- 172.21.15.114:0/3832278171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e41585b0 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e9ca6700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 0x7f40e4159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40e94a5700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f40d40118d0 tx=0x7f40d4011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004220 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4004380 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e4158890 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40036a0 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.549+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e4160270 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.550+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e40753f0 con 0x7f40d0004850 2023-11-24T06:27:27.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.551+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f40d4003800 con 0x7f40d0004850 2023-11-24T06:27:27.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.551+0000 7f40d9ffb700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 0x7f40c806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:27.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.551+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f40d4089ec0 con 0x7f40d0004850 2023-11-24T06:27:27.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.551+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 0x7f40c806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:27.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.551+0000 7f40e8ca4700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 0x7f40c806d600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40e00018b0 tx=0x7f40e000a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:27.558 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.555+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40d4054b40 con 0x7f40d0004850 2023-11-24T06:27:27.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.821+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40e4158ad0 con 0x7f40d0004850 2023-11-24T06:27:27.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.822+0000 7f40d9ffb700 1 -- 172.21.15.114:0/3832278171 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f40e4158ad0 con 0x7f40d0004850 2023-11-24T06:27:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:27.825 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 msgr2=0x7f40c806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 0x7f40c806d600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40e00018b0 tx=0x7f40e000a000 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 msgr2=0x7f40e415fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f40d40118d0 tx=0x7f40d4011c90 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 shutdown_connections 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f40c806b140 0x7f40c806d600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e4158db0 0x7f40e4159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d0004850 0x7f40e415fd30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 --2- 172.21.15.114:0/3832278171 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f40e4157130 0x7f40e41575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:27.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 >> 172.21.15.114:0/3832278171 conn(0x7f40e4071b90 msgr2=0x7f40e40af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:27.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 shutdown_connections 2023-11-24T06:27:27.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:27.824+0000 7f40eb709700 1 -- 172.21.15.114:0/3832278171 wait complete. 2023-11-24T06:27:27.828 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:27.839 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-24T06:27:27.977 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2023-11-24T06:27:27.977 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-24T06:27:27.978 DEBUG:tasks.ceph_test_case:wait_until_true: success in 70s and 0 retries 2023-11-24T06:27:27.978 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:27:27.978 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:28.020 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:28.073 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:27:28.073 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:28.073 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:28.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:27:28.091 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:27:28.092 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:28.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:27:28.112 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:28.112 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:28.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:27:28.149 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:27:28.149 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:28.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:27:28.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:28.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.306+0000 7f958fb06700 1 Processor -- start 2023-11-24T06:27:28.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.306+0000 7f958fb06700 1 -- start start 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958fb06700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9588144660 0x7f95880b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958fb06700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958815f520 con 0x7f958815ea60 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958fb06700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958815f800 con 0x7f958806c710 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958fb06700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880b4230 con 0x7f9588144660 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958d0a1700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f958815ea60 0x7f958806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57268/0 (socket says 172.21.15.114:57268) 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.307+0000 7f958d0a1700 1 -- 172.21.15.114:0/3958266235 learned_addr learned my addr 172.21.15.114:0/3958266235 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d8a2700 1 --2- 172.21.15.114:0/3958266235 >> v2:172.21.15.172:3300/0 conn(0x7f9588144660 0x7f95880b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133654250 0 0) 0x7f958815f520 con 0x7f958815ea60 2023-11-24T06:27:28.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95740035b0 con 0x7f958815ea60 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159195932 0 0) 0x7f958815f800 con 0x7f958806c710 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f958815f520 con 0x7f958806c710 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362557907 0 0) 0x7f95740035b0 con 0x7f958815ea60 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 >> v2:172.21.15.172:3300/0 conn(0x7f9588144660 msgr2=0x7f95880b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 --2- 172.21.15.114:0/3958266235 >> v2:172.21.15.172:3300/0 conn(0x7f9588144660 0x7f95880b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 >> v1:172.21.15.172:6789/0 conn(0x7f958806c710 legacy=0x7f958806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880b3e50 con 0x7f958815ea60 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f9584001670 con 0x7f958815ea60 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 >> v1:172.21.15.114:6789/0 conn(0x7f958815ea60 legacy=0x7f958806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9574003b90 0x7f9574003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f9574004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9574005210 0x7f957400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880b4230 con 0x7f9574004850 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957400c140 con 0x7f9574005210 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957400c300 con 0x7f9574003b90 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f9574004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d8a2700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9574003b90 0x7f9574003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958e0a3700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9574005210 0x7f957400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 -- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9574003b90 msgr2=0x7f9574003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9574003b90 0x7f9574003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 -- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9574005210 msgr2=0x7f957400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9574005210 0x7f957400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958e0a3700 1 -- 172.21.15.114:0/3958266235 reap_dead start 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.308+0000 7f958d0a1700 1 -- 172.21.15.114:0/3958266235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740035b0 con 0x7f9574004850 2023-11-24T06:27:28.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f9574004cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f958400c210 tx=0x7f9574005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5154c443e5dba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400cd50 con 0x7f9574004850 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9584001af0 con 0x7f9574004850 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958c8a0700 1 -- 172.21.15.114:0/3958266235 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584005ba0 con 0x7f9574004850 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 -- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 msgr2=0x7f9574004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f9574004cd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f958400c210 tx=0x7f9574005880 comp rx=0 tx=0).stop 2023-11-24T06:27:28.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 -- 172.21.15.114:0/3958266235 shutdown_connections 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 --2- 172.21.15.114:0/3958266235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f9574004cd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 -- 172.21.15.114:0/3958266235 >> 172.21.15.114:0/3958266235 conn(0x7f9588071ac0 msgr2=0x7f9588071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 -- 172.21.15.114:0/3958266235 shutdown_connections 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.309+0000 7f958fb06700 1 -- 172.21.15.114:0/3958266235 wait complete. 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 Processor -- start 2023-11-24T06:27:28.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 -- start start 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f958813c090 0x7f958815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f958815db50 0x7f958814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958815dfd0 con 0x7f9574004850 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958813ea20 con 0x7f958813c090 2023-11-24T06:27:28.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958fb06700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958813ed00 con 0x7f958815db50 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958d8a2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958d8a2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41570/0 (socket says 172.21.15.114:41570) 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958d8a2700 1 -- 172.21.15.114:0/3262537826 learned_addr learned my addr 172.21.15.114:0/3262537826 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.311+0000 7f958e0a3700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f958815db50 0x7f958814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f958813c090 0x7f958815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 -- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f958815db50 msgr2=0x7f958814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f958815db50 0x7f958814d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 -- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f958813c090 msgr2=0x7f958815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f958813c090 0x7f958815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 -- 172.21.15.114:0/3262537826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958813efe0 con 0x7f9574004850 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958d8a2700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9578011aa0 tx=0x7f9578011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578011ed0 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957800f940 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800fd10 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958813f2c0 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.312+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958813c510 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.313+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9588068980 con 0x7f9574004850 2023-11-24T06:27:28.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.313+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9578014440 con 0x7f9574004850 2023-11-24T06:27:28.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.314+0000 7f957e7fc700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 0x7f956c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.314+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9578089fc0 con 0x7f9574004850 2023-11-24T06:27:28.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.314+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 0x7f956c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.314+0000 7f958d0a1700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 0x7f956c06d600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9588151cd0 tx=0x7f958400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.318+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95780bce90 con 0x7f9574004850 2023-11-24T06:27:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.554+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9588067f60 con 0x7f9574004850 2023-11-24T06:27:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.555+0000 7f957e7fc700 1 -- 172.21.15.114:0/3262537826 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f9578022070 con 0x7f9574004850 2023-11-24T06:27:28.558 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:28.558 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 msgr2=0x7f956c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 0x7f956c06d600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9588151cd0 tx=0x7f958400a000 comp rx=0 tx=0).stop 2023-11-24T06:27:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 msgr2=0x7f958813f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9578011aa0 tx=0x7f9578011db0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 shutdown_connections 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f956c06b140 0x7f956c06d600 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f958815db50 0x7f958814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9574004850 0x7f958813f7e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 --2- 172.21.15.114:0/3262537826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f958813c090 0x7f958815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 >> 172.21.15.114:0/3262537826 conn(0x7f9588071ac0 msgr2=0x7f95880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 shutdown_connections 2023-11-24T06:27:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.557+0000 7f958fb06700 1 -- 172.21.15.114:0/3262537826 wait complete. 2023-11-24T06:27:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:28.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:28.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.692+0000 7fd193e4c700 1 Processor -- start 2023-11-24T06:27:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.692+0000 7fd193e4c700 1 -- start start 2023-11-24T06:27:28.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd193e4c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd18c15e080 0x7fd18c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd193e4c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd18c0af4d0 0x7fd18c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd193e4c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0ae430 con 0x7fd18c06c000 2023-11-24T06:27:28.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd193e4c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0af0b0 con 0x7fd18c0af4d0 2023-11-24T06:27:28.702 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd193e4c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0b05a0 con 0x7fd18c15e080 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd18c0af4d0 0x7fd18c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1923e9700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd18c06c000 0x7fd18c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57288/0 (socket says 172.21.15.114:57288) 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1923e9700 1 -- 172.21.15.114:0/1165095470 learned_addr learned my addr 172.21.15.114:0/1165095470 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd191be8700 1 --2- 172.21.15.114:0/1165095470 >> v2:172.21.15.172:3300/0 conn(0x7fd18c15e080 0x7fd18c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 -- 172.21.15.114:0/1165095470 >> v2:172.21.15.172:3300/0 conn(0x7fd18c15e080 msgr2=0x7fd18c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 --2- 172.21.15.114:0/1165095470 >> v2:172.21.15.172:3300/0 conn(0x7fd18c15e080 0x7fd18c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 -- 172.21.15.114:0/1165095470 >> v1:172.21.15.114:6789/0 conn(0x7fd18c06c000 legacy=0x7fd18c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd190be6700 1 -- 172.21.15.114:0/1165095470 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788965028 0 0) 0x7fd18c0ae430 con 0x7fd18c06c000 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 -- 172.21.15.114:0/1165095470 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0ae710 con 0x7fd18c0af4d0 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd1913e7700 1 --2- 172.21.15.114:0/1165095470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c0af4d0 0x7fd18c0afb90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd18c072a90 tx=0x7fd17c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a4e42c5a6fcfd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.693+0000 7fd190be6700 1 -- 172.21.15.114:0/1165095470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c011830 con 0x7fd18c0af4d0 2023-11-24T06:27:28.703 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd190be6700 1 -- 172.21.15.114:0/1165095470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c011990 con 0x7fd18c0af4d0 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd190be6700 1 -- 172.21.15.114:0/1165095470 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c011d20 con 0x7fd18c0af4d0 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 -- 172.21.15.114:0/1165095470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c0af4d0 msgr2=0x7fd18c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 --2- 172.21.15.114:0/1165095470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c0af4d0 0x7fd18c0afb90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd18c072a90 tx=0x7fd17c009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 -- 172.21.15.114:0/1165095470 shutdown_connections 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 --2- 172.21.15.114:0/1165095470 >> v2:172.21.15.172:3300/0 conn(0x7fd18c15e080 0x7fd18c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 --2- 172.21.15.114:0/1165095470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c0af4d0 0x7fd18c0afb90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 -- 172.21.15.114:0/1165095470 >> 172.21.15.114:0/1165095470 conn(0x7fd18c071b90 msgr2=0x7fd18c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 -- 172.21.15.114:0/1165095470 shutdown_connections 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.694+0000 7fd193e4c700 1 -- 172.21.15.114:0/1165095470 wait complete. 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.695+0000 7fd193e4c700 1 Processor -- start 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.695+0000 7fd193e4c700 1 -- start start 2023-11-24T06:27:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd18c0af4d0 0x7fd18c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd18c15e080 0x7fd18c141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c1538f0 con 0x7fd18c06c000 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c161350 con 0x7fd18c15e080 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd193e4c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c154170 con 0x7fd18c0af4d0 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41582/0 (socket says 172.21.15.114:41582) 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 -- 172.21.15.114:0/470936861 learned_addr learned my addr 172.21.15.114:0/470936861 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 -- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd18c0af4d0 msgr2=0x7fd18c1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd18c0af4d0 0x7fd18c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 -- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd18c15e080 msgr2=0x7fd18c141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd18c15e080 0x7fd18c141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.705 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.696+0000 7fd191be8700 1 -- 172.21.15.114:0/470936861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0094a0 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd191be8700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880125f0 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd188012750 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c161070 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188012b00 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.697+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c153bd0 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.698+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c0753f0 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.698+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd188012c60 con 0x7fd18c06c000 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.699+0000 7fd1827fc700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 0x7fd17006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.699+0000 7fd1913e7700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 0x7fd17006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.699+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd188089df0 con 0x7fd18c06c000 2023-11-24T06:27:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.699+0000 7fd1913e7700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 0x7fd17006d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd18c072a90 tx=0x7fd17c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.703+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd188054b00 con 0x7fd18c06c000 2023-11-24T06:27:28.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.895+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd18c0d09b0 con 0x7fd18c06c000 2023-11-24T06:27:28.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.896+0000 7fd1827fc700 1 -- 172.21.15.114:0/470936861 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fd188058120 con 0x7fd18c06c000 2023-11-24T06:27:28.899 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:28.899 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:59.863101+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:28.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 msgr2=0x7fd17006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.901 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 0x7fd17006d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd18c072a90 tx=0x7fd17c010000 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 msgr2=0x7fd18c1426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dbc0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 shutdown_connections 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fd17006b110 0x7fd17006d5d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd18c15e080 0x7fd18c141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd18c06c000 0x7fd18c1426e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 --2- 172.21.15.114:0/470936861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd18c0af4d0 0x7fd18c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 >> 172.21.15.114:0/470936861 conn(0x7fd18c071b90 msgr2=0x7fd18c073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 shutdown_connections 2023-11-24T06:27:28.902 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:28.899+0000 7fd193e4c700 1 -- 172.21.15.114:0/470936861 wait complete. 2023-11-24T06:27:28.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:25:48.353169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:25:48.873132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:28.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.034+0000 7fdc3fb72700 1 Processor -- start 2023-11-24T06:27:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.034+0000 7fdc3fb72700 1 -- start start 2023-11-24T06:27:29.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3fb72700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc3815e290 0x7fdc380ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3fb72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 0x7fdc3806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3fb72700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3806e400 con 0x7fdc380aedc0 2023-11-24T06:27:29.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3fb72700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3806e630 con 0x7fdc3815e290 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3fb72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3806e910 con 0x7fdc380ad870 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3d10d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc3815e290 0x7fdc380ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3d10d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdc3815e290 0x7fdc380ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41592/0 (socket says 172.21.15.114:41592) 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3d10d700 1 -- 172.21.15.114:0/1238422897 learned_addr learned my addr 172.21.15.114:0/1238422897 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.035+0000 7fdc3d90e700 1 --1- 172.21.15.114:0/1238422897 >> v1:172.21.15.114:6789/0 conn(0x7fdc380aedc0 0x7fdc380af1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57298/0 (socket says 172.21.15.114:57298) 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3e10f700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 0x7fdc3806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3d10d700 1 -- 172.21.15.114:0/1238422897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 msgr2=0x7fdc3806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3d10d700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 0x7fdc3806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3d10d700 1 -- 172.21.15.114:0/1238422897 >> v1:172.21.15.114:6789/0 conn(0x7fdc380aedc0 legacy=0x7fdc380af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3d10d700 1 -- 172.21.15.114:0/1238422897 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3806ebf0 con 0x7fdc3815e290 2023-11-24T06:27:29.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3c90c700 1 -- 172.21.15.114:0/1238422897 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768504474 0 0) 0x7fdc3806e400 con 0x7fdc380aedc0 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3e10f700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 0x7fdc3806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3d10d700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc3815e290 0x7fdc380ae790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098f0 tx=0x7fdc34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704d352a7887d939 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3c90c700 1 -- 172.21.15.114:0/1238422897 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400b8a0 con 0x7fdc3815e290 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3c90c700 1 -- 172.21.15.114:0/1238422897 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3400ba00 con 0x7fdc3815e290 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.036+0000 7fdc3c90c700 1 -- 172.21.15.114:0/1238422897 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400bd90 con 0x7fdc3815e290 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 -- 172.21.15.114:0/1238422897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc3815e290 msgr2=0x7fdc380ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc3815e290 0x7fdc380ae790 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098f0 tx=0x7fdc34009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 -- 172.21.15.114:0/1238422897 shutdown_connections 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380ad870 0x7fdc3806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/1238422897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc3815e290 0x7fdc380ae790 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 -- 172.21.15.114:0/1238422897 >> 172.21.15.114:0/1238422897 conn(0x7fdc380b2050 msgr2=0x7fdc380b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 -- 172.21.15.114:0/1238422897 shutdown_connections 2023-11-24T06:27:29.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.037+0000 7fdc3fb72700 1 -- 172.21.15.114:0/1238422897 wait complete. 2023-11-24T06:27:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.038+0000 7fdc3fb72700 1 Processor -- start 2023-11-24T06:27:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.038+0000 7fdc3fb72700 1 -- start start 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc380ad870 0x7fdc381f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc3815e290 0x7fdc38150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381f7f70 con 0x7fdc380ad870 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38151220 con 0x7fdc3815e290 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3fb72700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381f49b0 con 0x7fdc380aedc0 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36142/0 (socket says 172.21.15.114:36142) 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3e10f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc3815e290 0x7fdc38150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 -- 172.21.15.114:0/3270439221 learned_addr learned my addr 172.21.15.114:0/3270439221 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d90e700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc380ad870 0x7fdc381f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 -- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc3815e290 msgr2=0x7fdc38150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc3815e290 0x7fdc38150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 -- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc380ad870 msgr2=0x7fdc381f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc380ad870 0x7fdc381f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.039+0000 7fdc3d10d700 1 -- 172.21.15.114:0/3270439221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009550 con 0x7fdc380aedc0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc3d10d700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc38062340 tx=0x7fdc34004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34004450 con 0x7fdc380aedc0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340045b0 con 0x7fdc380aedc0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340038a0 con 0x7fdc380aedc0 2023-11-24T06:27:29.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc381f4110 con 0x7fdc380aedc0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.040+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381f46d0 con 0x7fdc380aedc0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.041+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc38151ca0 con 0x7fdc380aedc0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.041+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 7) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdc34003a00 con 0x7fdc380aedc0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.042+0000 7fdc2e7fc700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 0x7fdc1c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.042+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdc3408b2c0 con 0x7fdc380aedc0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.042+0000 7fdc3d90e700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 0x7fdc1c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.042+0000 7fdc3d90e700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 0x7fdc1c06d5d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007fd0 tx=0x7fdc28007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.048 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.046+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc34056180 con 0x7fdc380aedc0 2023-11-24T06:27:29.072 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.070+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fdc34056550 con 0x7fdc380aedc0 2023-11-24T06:27:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.239+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc38068d90 con 0x7fdc380aedc0 2023-11-24T06:27:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.240+0000 7fdc2e7fc700 1 -- 172.21.15.114:0/3270439221 <== mon.2 v2:172.21.15.172:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc340835d0 con 0x7fdc380aedc0 2023-11-24T06:27:29.243 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:27:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 msgr2=0x7fdc1c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 0x7fdc1c06d5d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007fd0 tx=0x7fdc28007de0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 msgr2=0x7fdc381f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc38062340 tx=0x7fdc34004750 comp rx=0 tx=0).stop 2023-11-24T06:27:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 shutdown_connections 2023-11-24T06:27:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc1c06b110 0x7fdc1c06d5d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc3815e290 0x7fdc38150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc380ad870 0x7fdc381f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 --2- 172.21.15.114:0/3270439221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc380aedc0 0x7fdc381f7a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 >> 172.21.15.114:0/3270439221 conn(0x7fdc380b2050 msgr2=0x7fdc38061aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 shutdown_connections 2023-11-24T06:27:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.242+0000 7fdc3fb72700 1 -- 172.21.15.114:0/3270439221 wait complete. 2023-11-24T06:27:29.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:29.380 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.378+0000 7fdfedd99700 1 Processor -- start 2023-11-24T06:27:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.378+0000 7fdfedd99700 1 -- start start 2023-11-24T06:27:29.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfedd99700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdfe80ceb40 0x7fdfe806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfedd99700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe806ba50 con 0x7fdfe815e080 2023-11-24T06:27:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfedd99700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe806b120 con 0x7fdfe80b4310 2023-11-24T06:27:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfedd99700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe80d1200 con 0x7fdfe80ceb40 2023-11-24T06:27:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdfe80ceb40 0x7fdfe806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdfe80ceb40 0x7fdfe806bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41606/0 (socket says 172.21.15.114:41606) 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 -- 172.21.15.114:0/3530751670 learned_addr learned my addr 172.21.15.114:0/3530751670 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe77fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdfe815e080 0x7fdfe80d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57312/0 (socket says 172.21.15.114:57312) 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 -- 172.21.15.114:0/3530751670 >> v1:172.21.15.172:6789/0 conn(0x7fdfe80b4310 legacy=0x7fdfe80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 -- 172.21.15.114:0/3530751670 >> v1:172.21.15.114:6789/0 conn(0x7fdfe815e080 legacy=0x7fdfe80d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 -- 172.21.15.114:0/3530751670 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe806b400 con 0x7fdfe80ceb40 2023-11-24T06:27:29.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe67fc700 1 -- 172.21.15.114:0/3530751670 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354799029 0 0) 0x7fdfe806ba50 con 0x7fdfe815e080 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe7fff700 1 --2- 172.21.15.114:0/3530751670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80ceb40 0x7fdfe806bee0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e50 tx=0x7fdfe000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01d698ab585c77f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe67fc700 1 -- 172.21.15.114:0/3530751670 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000e070 con 0x7fdfe80ceb40 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe67fc700 1 -- 172.21.15.114:0/3530751670 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0003a20 con 0x7fdfe80ceb40 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.379+0000 7fdfe67fc700 1 -- 172.21.15.114:0/3530751670 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003dd0 con 0x7fdfe80ceb40 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 -- 172.21.15.114:0/3530751670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80ceb40 msgr2=0x7fdfe806bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 --2- 172.21.15.114:0/3530751670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80ceb40 0x7fdfe806bee0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e50 tx=0x7fdfe000b330 comp rx=0 tx=0).stop 2023-11-24T06:27:29.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 -- 172.21.15.114:0/3530751670 shutdown_connections 2023-11-24T06:27:29.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 --2- 172.21.15.114:0/3530751670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80ceb40 0x7fdfe806bee0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 -- 172.21.15.114:0/3530751670 >> 172.21.15.114:0/3530751670 conn(0x7fdfe80b3780 msgr2=0x7fdfe80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 -- 172.21.15.114:0/3530751670 shutdown_connections 2023-11-24T06:27:29.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.380+0000 7fdfedd99700 1 -- 172.21.15.114:0/3530751670 wait complete. 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.381+0000 7fdfedd99700 1 Processor -- start 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.381+0000 7fdfedd99700 1 -- start start 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ceb40 0x7fdfe8160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe815e080 0x7fdfe81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.394 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80b4e50 con 0x7fdfe80b4310 2023-11-24T06:27:29.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c9f20 con 0x7fdfe815e080 2023-11-24T06:27:29.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfedd99700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe815c670 con 0x7fdfe80ceb40 2023-11-24T06:27:29.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41608/0 (socket says 172.21.15.114:41608) 2023-11-24T06:27:29.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 -- 172.21.15.114:0/4057333390 learned_addr learned my addr 172.21.15.114:0/4057333390 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:29.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 -- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ceb40 msgr2=0x7fdfe8160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.397 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ceb40 0x7fdfe8160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 -- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe815e080 msgr2=0x7fdfe81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe815e080 0x7fdfe81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 -- 172.21.15.114:0/4057333390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009b30 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfe77fe700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c550 tx=0x7fdfd800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.382+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004c80 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.383+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8004e00 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.383+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe80b7560 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.383+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8007310 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.383+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe80c9bb0 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.383+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe8068fb0 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.384+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fdfd8007470 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.384+0000 7fdfcffff700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 0x7fdfd406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.384+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdfd8089630 con 0x7fdfe80b4310 2023-11-24T06:27:29.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.384+0000 7fdfe6ffd700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 0x7fdfd406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.385+0000 7fdfe6ffd700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 0x7fdfd406d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdfe815cae0 tx=0x7fdfdc007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.389+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfd8054420 con 0x7fdfe80b4310 2023-11-24T06:27:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.580+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfe80b7ab0 con 0x7fdfe80b4310 2023-11-24T06:27:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.580+0000 7fdfcffff700 1 -- 172.21.15.114:0/4057333390 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfd8057a40 con 0x7fdfe80b4310 2023-11-24T06:27:29.583 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 msgr2=0x7fdfd406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 0x7fdfd406d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdfe815cae0 tx=0x7fdfdc007a40 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 msgr2=0x7fdfe80b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c550 tx=0x7fdfd800c910 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 shutdown_connections 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdfd406b080 0x7fdfd406d540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfe815e080 0x7fdfe81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfe80b4310 0x7fdfe80b4870 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 --2- 172.21.15.114:0/4057333390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfe80ceb40 0x7fdfe8160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 >> 172.21.15.114:0/4057333390 conn(0x7fdfe80b3780 msgr2=0x7fdfe80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 shutdown_connections 2023-11-24T06:27:29.586 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.582+0000 7fdfedd99700 1 -- 172.21.15.114:0/4057333390 wait complete. 2023-11-24T06:27:29.596 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:29.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 Processor -- start 2023-11-24T06:27:29.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 -- start start 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc501587f0 0x7fcc5006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc5015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5006b530 con 0x7fcc500b3c60 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc500ae5e0 con 0x7fcc501587f0 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.717+0000 7fcc563c1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5015be30 con 0x7fcc5015b370 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc5495e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcc500b3c60 0x7fcc500af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57328/0 (socket says 172.21.15.114:57328) 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc5495e700 1 -- 172.21.15.114:0/1865085235 learned_addr learned my addr 172.21.15.114:0/1865085235 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/1865085235 >> v2:172.21.15.172:3300/0 conn(0x7fcc501587f0 0x7fcc5006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327928224 0 0) 0x7fcc5006b530 con 0x7fcc500b3c60 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc5015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc380035b0 con 0x7fcc500b3c60 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104375787 0 0) 0x7fcc380035b0 con 0x7fcc500b3c60 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 msgr2=0x7fcc5015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc5015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 >> v2:172.21.15.172:3300/0 conn(0x7fcc501587f0 msgr2=0x7fcc5006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 --2- 172.21.15.114:0/1865085235 >> v2:172.21.15.172:3300/0 conn(0x7fcc501587f0 0x7fcc5006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5015c110 con 0x7fcc500b3c60 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fcc480019f0 con 0x7fcc500b3c60 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 >> v1:172.21.15.114:6789/0 conn(0x7fcc500b3c60 legacy=0x7fcc500af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc38003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc38004030 0x7fcc38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc38004d90 0x7fcc38005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500ae5e0 con 0x7fcc38004d90 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5015be30 con 0x7fcc38004030 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3800bb10 con 0x7fcc5015b370 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/1865085235 >> v2:172.21.15.172:3300/0 conn(0x7fcc501587f0 0x7fcc5006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc5495e700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc38004d90 0x7fcc38005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc38004030 0x7fcc38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.718+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc38003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 -- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 msgr2=0x7fcc38003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5015b370 0x7fcc38003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 -- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc38004030 msgr2=0x7fcc38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc38004030 0x7fcc38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 -- 172.21.15.114:0/1865085235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380035b0 con 0x7fcc38004d90 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc38004030 0x7fcc38004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc38004d90 0x7fcc38005210 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800a850 tx=0x7fcc38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eeaf7392c6e447d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48012600 con 0x7fcc38004d90 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48012760 con 0x7fcc38004d90 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc4effd700 1 -- 172.21.15.114:0/1865085235 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48012b10 con 0x7fcc38004d90 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 -- 172.21.15.114:0/1865085235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc38004d90 msgr2=0x7fcc38005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 --2- 172.21.15.114:0/1865085235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc38004d90 0x7fcc38005210 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800a850 tx=0x7fcc38005880 comp rx=0 tx=0).stop 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc5495e700 1 -- 172.21.15.114:0/1865085235 reap_dead start 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 -- 172.21.15.114:0/1865085235 shutdown_connections 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 -- 172.21.15.114:0/1865085235 >> 172.21.15.114:0/1865085235 conn(0x7fcc500d0bf0 msgr2=0x7fcc500d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 -- 172.21.15.114:0/1865085235 shutdown_connections 2023-11-24T06:27:29.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.719+0000 7fcc563c1700 1 -- 172.21.15.114:0/1865085235 wait complete. 2023-11-24T06:27:29.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.721+0000 7fcc563c1700 1 Processor -- start 2023-11-24T06:27:29.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.721+0000 7fcc563c1700 1 -- start start 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc50157240 0x7fcc50143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 0x7fcc5006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50143070 con 0x7fcc50142640 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50143350 con 0x7fcc50157240 2023-11-24T06:27:29.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50143630 con 0x7fcc5009e230 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41628/0 (socket says 172.21.15.114:41628) 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 -- 172.21.15.114:0/863639800 learned_addr learned my addr 172.21.15.114:0/863639800 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc50157240 0x7fcc50143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc5495e700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 0x7fcc5006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 -- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 msgr2=0x7fcc5006e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 0x7fcc5006e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 -- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc50157240 msgr2=0x7fcc50143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc50157240 0x7fcc50143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 -- 172.21.15.114:0/863639800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50143910 con 0x7fcc50142640 2023-11-24T06:27:29.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc5495e700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 0x7fcc5006e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4ffff700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcc40010890 tx=0x7fcc40010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000e860 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4000c050 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5009e6c0 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000ec00 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.722+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5006f0d0 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.723+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc501431b0 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.724+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcc4000ed60 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.724+0000 7fcc4cff9700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 0x7fcc3c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.724+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcc4008a260 con 0x7fcc50142640 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.724+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 0x7fcc3c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:29.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.724+0000 7fcc4f7fe700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 0x7fcc3c06d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400e880 tx=0x7fcc44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:29.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.728+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc501431b0 con 0x7fcc50142640 2023-11-24T06:27:29.920 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.918+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc501431b0 con 0x7fcc50142640 2023-11-24T06:27:29.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.918+0000 7fcc4cff9700 1 -- 172.21.15.114:0/863639800 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc501431b0 con 0x7fcc50142640 2023-11-24T06:27:29.921 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:27:29.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 msgr2=0x7fcc3c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 0x7fcc3c06d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400e880 tx=0x7fcc44008040 comp rx=0 tx=0).stop 2023-11-24T06:27:29.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 msgr2=0x7fcc50142a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:29.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcc40010890 tx=0x7fcc40010c50 comp rx=0 tx=0).stop 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 shutdown_connections 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcc3c06b080 0x7fcc3c06d540 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc5009e230 0x7fcc5006e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc50142640 0x7fcc50142a20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 --2- 172.21.15.114:0/863639800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc50157240 0x7fcc50143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.920+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 >> 172.21.15.114:0/863639800 conn(0x7fcc500d0bf0 msgr2=0x7fcc50156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.921+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 shutdown_connections 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:29.921+0000 7fcc563c1700 1 -- 172.21.15.114:0/863639800 wait complete. 2023-11-24T06:27:29.935 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:30.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.052+0000 7fc9f8c52700 1 Processor -- start 2023-11-24T06:27:30.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.052+0000 7fc9f8c52700 1 -- start start 2023-11-24T06:27:30.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 0x7fc9f40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f40af4d0 0x7fc9f40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9f406c000 0x7fc9f406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f40af0b0 con 0x7fc9f406c000 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f406c9f0 con 0x7fc9f415e080 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f8c52700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40b05a0 con 0x7fc9f40af4d0 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f2d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9f406c000 0x7fc9f406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f2d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9f406c000 0x7fc9f406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41640/0 (socket says 172.21.15.114:41640) 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.053+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2278185118 learned_addr learned my addr 172.21.15.114:0/2278185118 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f259c700 1 --2- 172.21.15.114:0/2278185118 >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 0x7fc9f40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f1d9b700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f40af4d0 0x7fc9f40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2278185118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f40af4d0 msgr2=0x7fc9f40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f40af4d0 0x7fc9f40afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2278185118 >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 msgr2=0x7fc9f40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2278185118 >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 0x7fc9f40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2278185118 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f40ae430 con 0x7fc9f406c000 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f259c700 1 --2- 172.21.15.114:0/2278185118 >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 0x7fc9f40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f406c000 0x7fc9f406c4b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4072a90 tx=0x7fc9ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecec686575df049 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f159a700 1 -- 172.21.15.114:0/2278185118 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00e070 con 0x7fc9f406c000 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f159a700 1 -- 172.21.15.114:0/2278185118 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec00b980 con 0x7fc9f406c000 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f159a700 1 -- 172.21.15.114:0/2278185118 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00bd30 con 0x7fc9f406c000 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2278185118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f406c000 msgr2=0x7fc9f406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.054+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f406c000 0x7fc9f406c4b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4072a90 tx=0x7fc9ec00b330 comp rx=0 tx=0).stop 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2278185118 shutdown_connections 2023-11-24T06:27:30.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f406c000 0x7fc9f406c4b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2278185118 >> v2:172.21.15.172:3300/0 conn(0x7fc9f415e080 0x7fc9f40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2278185118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f40af4d0 0x7fc9f40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2278185118 >> 172.21.15.114:0/2278185118 conn(0x7fc9f4071b90 msgr2=0x7fc9f4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:30.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2278185118 shutdown_connections 2023-11-24T06:27:30.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.055+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2278185118 wait complete. 2023-11-24T06:27:30.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.056+0000 7fc9f8c52700 1 Processor -- start 2023-11-24T06:27:30.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.056+0000 7fc9f8c52700 1 -- start start 2023-11-24T06:27:30.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f406c000 0x7fc9f4153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 0x7fc9f40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4157d40 con 0x7fc9f415e080 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f4157710 con 0x7fc9f40af4d0 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40d0220 con 0x7fc9f406c000 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41646/0 (socket says 172.21.15.114:41646) 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2797536353 learned_addr learned my addr 172.21.15.114:0/2797536353 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f259c700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f406c000 0x7fc9f4153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f1d9b700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 0x7fc9f40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f406c000 msgr2=0x7fc9f4153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f406c000 0x7fc9f4153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 msgr2=0x7fc9f40cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 0x7fc9f40cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 -- 172.21.15.114:0/2797536353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec009b30 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f1d9b700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 0x7fc9f40cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f2d9d700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc9f409f040 tx=0x7fc9ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec004250 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0043b0 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f40d0500 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.057+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec004760 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.058+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f4157360 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.058+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f40753f0 con 0x7fc9f415e080 2023-11-24T06:27:30.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.059+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc9ec01c440 con 0x7fc9f415e080 2023-11-24T06:27:30.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.059+0000 7fc9e2ffd700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 0x7fc9d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.059+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc9ec08afd0 con 0x7fc9f415e080 2023-11-24T06:27:30.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.059+0000 7fc9f259c700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 0x7fc9d406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.060+0000 7fc9f259c700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 0x7fc9d406d570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8006fd0 tx=0x7fc9e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.064+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9ec055cc0 con 0x7fc9f415e080 2023-11-24T06:27:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.298+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9f4157e80 con 0x7fc9f415e080 2023-11-24T06:27:30.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.299+0000 7fc9e2ffd700 1 -- 172.21.15.114:0/2797536353 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fc9f4157e80 con 0x7fc9f415e080 2023-11-24T06:27:30.301 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:30.302 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:30.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 msgr2=0x7fc9d406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 0x7fc9d406d570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8006fd0 tx=0x7fc9e8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:30.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 msgr2=0x7fc9f40cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc9f409f040 tx=0x7fc9ec004ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 shutdown_connections 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc9d406b0b0 0x7fc9d406d570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9f415e080 0x7fc9f40cd600 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9f406c000 0x7fc9f4153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 --2- 172.21.15.114:0/2797536353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9f40af4d0 0x7fc9f40cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 >> 172.21.15.114:0/2797536353 conn(0x7fc9f4071b90 msgr2=0x7fc9f415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 shutdown_connections 2023-11-24T06:27:30.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.301+0000 7fc9f8c52700 1 -- 172.21.15.114:0/2797536353 wait complete. 2023-11-24T06:27:30.305 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:30.315 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.431+0000 7fcb50e17700 1 Processor -- start 2023-11-24T06:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.431+0000 7fcb50e17700 1 -- start start 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.432+0000 7fcb50e17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 0x7fcb4c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.432+0000 7fcb50e17700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 0x7fcb4c0d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.432+0000 7fcb50e17700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c0cd060 con 0x7fcb4c0d2020 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.432+0000 7fcb50e17700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f940 con 0x7fcb4c0d0740 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.432+0000 7fcb50e17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0cdec0 con 0x7fcb4c15f560 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4ad9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcb4c0d2020 0x7fcb4c0ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57362/0 (socket says 172.21.15.114:57362) 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4ad9d700 1 -- 172.21.15.114:0/851619535 learned_addr learned my addr 172.21.15.114:0/851619535 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/851619535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 0x7fcb4c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb49d9b700 1 --2- 172.21.15.114:0/851619535 >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 0x7fcb4c0d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4959a700 1 -- 172.21.15.114:0/851619535 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237891976 0 0) 0x7fcb4c0cd060 con 0x7fcb4c0d2020 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4959a700 1 -- 172.21.15.114:0/851619535 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb340035b0 con 0x7fcb4c0d2020 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4a59c700 1 -- 172.21.15.114:0/851619535 >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 msgr2=0x7fcb4c0d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/851619535 >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 0x7fcb4c0d1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4a59c700 1 -- 172.21.15.114:0/851619535 >> v1:172.21.15.114:6789/0 conn(0x7fcb4c0d2020 legacy=0x7fcb4c0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb4a59c700 1 -- 172.21.15.114:0/851619535 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c0b1420 con 0x7fcb4c15f560 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.433+0000 7fcb49d9b700 1 --2- 172.21.15.114:0/851619535 >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 0x7fcb4c0d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/851619535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 0x7fcb4c0d0200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009840 tx=0x7fcb40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b25593ff581ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb4959a700 1 -- 172.21.15.114:0/851619535 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40011830 con 0x7fcb4c15f560 2023-11-24T06:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb4959a700 1 -- 172.21.15.114:0/851619535 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb40011990 con 0x7fcb4c15f560 2023-11-24T06:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb4959a700 1 -- 172.21.15.114:0/851619535 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40011d00 con 0x7fcb4c15f560 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 -- 172.21.15.114:0/851619535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 msgr2=0x7fcb4c0d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 --2- 172.21.15.114:0/851619535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 0x7fcb4c0d0200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009840 tx=0x7fcb40009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 -- 172.21.15.114:0/851619535 shutdown_connections 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 --2- 172.21.15.114:0/851619535 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c15f560 0x7fcb4c0d0200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 --2- 172.21.15.114:0/851619535 >> v2:172.21.15.172:3300/0 conn(0x7fcb4c0d0740 0x7fcb4c0d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 -- 172.21.15.114:0/851619535 >> 172.21.15.114:0/851619535 conn(0x7fcb4c09aa00 msgr2=0x7fcb4c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 -- 172.21.15.114:0/851619535 shutdown_connections 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.434+0000 7fcb50e17700 1 -- 172.21.15.114:0/851619535 wait complete. 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 Processor -- start 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 -- start start 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c0d0740 0x7fcb4c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c0d2020 0x7fcb4c1e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c06c940 con 0x7fcb4c15f560 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0c67d0 con 0x7fcb4c0d0740 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.436+0000 7fcb50e17700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c06cb90 con 0x7fcb4c0d2020 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41658/0 (socket says 172.21.15.114:41658) 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 -- 172.21.15.114:0/3430878823 learned_addr learned my addr 172.21.15.114:0/3430878823 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb49d9b700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c0d2020 0x7fcb4c1e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c0d0740 0x7fcb4c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 -- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c0d2020 msgr2=0x7fcb4c1e4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c0d2020 0x7fcb4c1e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 -- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c0d0740 msgr2=0x7fcb4c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c0d0740 0x7fcb4c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 -- 172.21.15.114:0/3430878823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0094a0 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb4ad9d700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400cb70 tx=0x7fcb4400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400aa60 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4400abc0 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44012400 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb400094a0 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.437+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c0c6130 con 0x7fcb4c15f560 2023-11-24T06:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.438+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c06cf80 con 0x7fcb4c15f560 2023-11-24T06:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.439+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcb4401b3b0 con 0x7fcb4c15f560 2023-11-24T06:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.439+0000 7fcb3affd700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 0x7fcb2c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.439+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcb440895b0 con 0x7fcb4c15f560 2023-11-24T06:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.439+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 0x7fcb2c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.439+0000 7fcb4a59c700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 0x7fcb2c06d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb4001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.443+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb440543a0 con 0x7fcb4c15f560 2023-11-24T06:27:30.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.633+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb4c068f10 con 0x7fcb4c15f560 2023-11-24T06:27:30.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.634+0000 7fcb3affd700 1 -- 172.21.15.114:0/3430878823 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fcb440579c0 con 0x7fcb4c15f560 2023-11-24T06:27:30.638 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:30.638 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:59.863101+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 msgr2=0x7fcb2c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 0x7fcb2c06d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb4001c040 comp rx=0 tx=0).stop 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 msgr2=0x7fcb4c1e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400cb70 tx=0x7fcb4400ce80 comp rx=0 tx=0).stop 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 shutdown_connections 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcb2c06b080 0x7fcb2c06d540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c15f560 0x7fcb4c1e4fc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c0d0740 0x7fcb4c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 --2- 172.21.15.114:0/3430878823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb4c0d2020 0x7fcb4c1e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 >> 172.21.15.114:0/3430878823 conn(0x7fcb4c09aa00 msgr2=0x7fcb4c15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 shutdown_connections 2023-11-24T06:27:30.641 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.636+0000 7fcb50e17700 1 -- 172.21.15.114:0/3430878823 wait complete. 2023-11-24T06:27:30.652 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:27:30.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:30.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.770+0000 7f09e8926700 1 Processor -- start 2023-11-24T06:27:30.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.770+0000 7f09e8926700 1 -- start start 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e8926700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09e406c710 0x7f09e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e8926700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e40ae430 con 0x7f09e40af4d0 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e8926700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e40ae710 con 0x7f09e415e080 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e8926700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e40af0b0 con 0x7f09e406c710 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e1d9b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f09e406c710 0x7f09e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e2d9d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f09e40af4d0 0x7f09e406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57374/0 (socket says 172.21.15.114:57374) 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e2d9d700 1 -- 172.21.15.114:0/3468331080 learned_addr learned my addr 172.21.15.114:0/3468331080 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e1d9b700 1 -- 172.21.15.114:0/3468331080 >> v1:172.21.15.172:6789/0 conn(0x7f09e415e080 legacy=0x7f09e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e1d9b700 1 -- 172.21.15.114:0/3468331080 >> v1:172.21.15.114:6789/0 conn(0x7f09e40af4d0 legacy=0x7f09e406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e159a700 1 -- 172.21.15.114:0/3468331080 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158757238 0 0) 0x7f09e40ae430 con 0x7f09e40af4d0 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e1d9b700 1 -- 172.21.15.114:0/3468331080 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40ae9f0 con 0x7f09e406c710 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e1d9b700 1 --2- 172.21.15.114:0/3468331080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e406c710 0x7f09e40b5050 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009840 tx=0x7f09d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c7e50c69ff5e39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.771+0000 7f09e159a700 1 -- 172.21.15.114:0/3468331080 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004230 con 0x7f09e406c710 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e159a700 1 -- 172.21.15.114:0/3468331080 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d8004390 con 0x7f09e406c710 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e159a700 1 -- 172.21.15.114:0/3468331080 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d80036a0 con 0x7f09e406c710 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 -- 172.21.15.114:0/3468331080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e406c710 msgr2=0x7f09e40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 --2- 172.21.15.114:0/3468331080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e406c710 0x7f09e40b5050 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009840 tx=0x7f09d8009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 -- 172.21.15.114:0/3468331080 shutdown_connections 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 --2- 172.21.15.114:0/3468331080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e406c710 0x7f09e40b5050 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 -- 172.21.15.114:0/3468331080 >> 172.21.15.114:0/3468331080 conn(0x7f09e4071b90 msgr2=0x7f09e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 -- 172.21.15.114:0/3468331080 shutdown_connections 2023-11-24T06:27:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.772+0000 7f09e8926700 1 -- 172.21.15.114:0/3468331080 wait complete. 2023-11-24T06:27:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.773+0000 7f09e8926700 1 Processor -- start 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 -- start start 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09e406c710 0x7f09e40cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09e40af4d0 0x7f09e41f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40cc2a0 con 0x7f09e415e080 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e41f49b0 con 0x7f09e40af4d0 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e8926700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e414fa90 con 0x7f09e406c710 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e2d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41672/0 (socket says 172.21.15.114:41672) 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.774+0000 7f09e2d9d700 1 -- 172.21.15.114:0/199205757 learned_addr learned my addr 172.21.15.114:0/199205757 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 -- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09e406c710 msgr2=0x7f09e40cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09e406c710 0x7f09e40cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 -- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09e40af4d0 msgr2=0x7f09e41f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09e40af4d0 0x7f09e41f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:30.781 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 -- 172.21.15.114:0/199205757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80094a0 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e2d9d700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cca0 tx=0x7f09dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00cea0 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc00ade0 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e40cc4d0 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc007310 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.775+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e415f550 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.776+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e40753f0 con 0x7f09e415e080 2023-11-24T06:27:30.782 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.776+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f09dc007470 con 0x7f09e415e080 2023-11-24T06:27:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.777+0000 7f09d2ffd700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 0x7f09c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.777+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f09dc089580 con 0x7f09e415e080 2023-11-24T06:27:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.777+0000 7f09e259c700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 0x7f09c406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:30.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.777+0000 7f09e259c700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 0x7f09c406d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f09e406bde0 tx=0x7f09d4006c30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:30.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:30.781+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09dc054400 con 0x7f09e415e080 2023-11-24T06:27:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.013+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09e40b5970 con 0x7f09e415e080 2023-11-24T06:27:31.017 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.014+0000 7f09d2ffd700 1 -- 172.21.15.114:0/199205757 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f09dc057a20 con 0x7f09e415e080 2023-11-24T06:27:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:31.018 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 msgr2=0x7f09c406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 0x7f09c406d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f09e406bde0 tx=0x7f09d4006c30 comp rx=0 tx=0).stop 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 msgr2=0x7f09e414f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00cca0 tx=0x7f09dc00a340 comp rx=0 tx=0).stop 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 shutdown_connections 2023-11-24T06:27:31.020 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09c406b080 0x7f09c406d540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09e415e080 0x7f09e414f550 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09e406c710 0x7f09e40cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 --2- 172.21.15.114:0/199205757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09e40af4d0 0x7f09e41f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 >> 172.21.15.114:0/199205757 conn(0x7f09e4071b90 msgr2=0x7f09e40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 shutdown_connections 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.016+0000 7f09e8926700 1 -- 172.21.15.114:0/199205757 wait complete. 2023-11-24T06:27:31.021 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:31.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:31.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.149+0000 7f38ee007700 1 Processor -- start 2023-11-24T06:27:31.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 -- start start 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e815e080 0x7f38e80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e80ae430 con 0x7f38e80af4d0 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e80af0b0 con 0x7f38e815e080 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38ee007700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b05a0 con 0x7f38e806c710 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38e77fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e815e080 0x7f38e80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.150+0000 7f38e77fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e815e080 0x7f38e80b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41676/0 (socket says 172.21.15.114:41676) 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 -- 172.21.15.114:0/1770091443 learned_addr learned my addr 172.21.15.114:0/1770091443 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:31.156 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e7fff700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 -- 172.21.15.114:0/1770091443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 msgr2=0x7f38e80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 -- 172.21.15.114:0/1770091443 >> v1:172.21.15.172:6789/0 conn(0x7f38e80af4d0 legacy=0x7f38e806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 -- 172.21.15.114:0/1770091443 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80ae710 con 0x7f38e815e080 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e7fff700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e77fe700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e80b5050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009920 tx=0x7f38d8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750db86a5b578e99 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e67fc700 1 -- 172.21.15.114:0/1770091443 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8004230 con 0x7f38e815e080 2023-11-24T06:27:31.157 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e67fc700 1 -- 172.21.15.114:0/1770091443 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8004390 con 0x7f38e815e080 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.151+0000 7f38e67fc700 1 -- 172.21.15.114:0/1770091443 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8009e70 con 0x7f38e815e080 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 -- 172.21.15.114:0/1770091443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 msgr2=0x7f38e80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e80b5050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009920 tx=0x7f38d8009c30 comp rx=0 tx=0).stop 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 -- 172.21.15.114:0/1770091443 shutdown_connections 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 --2- 172.21.15.114:0/1770091443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e80b5050 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 -- 172.21.15.114:0/1770091443 >> 172.21.15.114:0/1770091443 conn(0x7f38e8071b90 msgr2=0x7f38e8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 -- 172.21.15.114:0/1770091443 shutdown_connections 2023-11-24T06:27:31.158 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.152+0000 7f38ee007700 1 -- 172.21.15.114:0/1770091443 wait complete. 2023-11-24T06:27:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.153+0000 7f38ee007700 1 Processor -- start 2023-11-24T06:27:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.153+0000 7f38ee007700 1 -- start start 2023-11-24T06:27:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 0x7f38e8145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8154440 con 0x7f38e815e080 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81534e0 con 0x7f38e80af4d0 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38ee007700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81f8330 con 0x7f38e806c710 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41686/0 (socket says 172.21.15.114:41686) 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 -- 172.21.15.114:0/2563216287 learned_addr learned my addr 172.21.15.114:0/2563216287 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e6ffd700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 0x7f38e8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e77fe700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 -- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 msgr2=0x7f38e8151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e8151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 -- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 msgr2=0x7f38e8145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 0x7f38e8145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 -- 172.21.15.114:0/2563216287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009550 con 0x7f38e815e080 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e6ffd700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 0x7f38e8145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38e7fff700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38e000cea0 tx=0x7f38e000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000ae50 con 0x7f38e815e080 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.154+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e0003a20 con 0x7f38e815e080 2023-11-24T06:27:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.155+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e8153d80 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.155+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0003dd0 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.155+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e8152510 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.156+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e80753f0 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.156+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f38e0003f30 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.156+0000 7f38dffff700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 0x7f38c806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.156+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38e0089d80 con 0x7f38e815e080 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.156+0000 7f38e77fe700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 0x7f38c806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.157+0000 7f38e77fe700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 0x7f38c806d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009870 tx=0x7f38d8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.161+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38e0054b70 con 0x7f38e815e080 2023-11-24T06:27:31.395 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.392+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38e80d09b0 con 0x7f38e815e080 2023-11-24T06:27:31.396 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.393+0000 7f38dffff700 1 -- 172.21.15.114:0/2563216287 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f38e0058190 con 0x7f38e815e080 2023-11-24T06:27:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:31.396 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:25:49.913212+0000","modified":"2023-11-24T06:25:52.939341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/4123842361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":4123842361},{"type":"v1","addr":"172.21.15.114:6837","nonce":4123842361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:27:31.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 msgr2=0x7f38c806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 0x7f38c806d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009870 tx=0x7f38d8008030 comp rx=0 tx=0).stop 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 msgr2=0x7f38e8157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38e000cea0 tx=0x7f38e000cf80 comp rx=0 tx=0).stop 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 shutdown_connections 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f38c806b080 0x7f38c806d540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e815e080 0x7f38e8157400 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38e806c710 0x7f38e8151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 --2- 172.21.15.114:0/2563216287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38e80af4d0 0x7f38e8145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 >> 172.21.15.114:0/2563216287 conn(0x7f38e8071b90 msgr2=0x7f38e80b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 shutdown_connections 2023-11-24T06:27:31.400 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.395+0000 7f38ee007700 1 -- 172.21.15.114:0/2563216287 wait complete. 2023-11-24T06:27:31.401 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2023-11-24T06:27:31.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:31.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.528+0000 7fae9752e700 1 Processor -- start 2023-11-24T06:27:31.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.528+0000 7fae9752e700 1 -- start start 2023-11-24T06:27:31.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae9752e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fae9006c710 0x7fae900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae9752e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae900ae430 con 0x7fae9015e080 2023-11-24T06:27:31.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae9752e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae900ae710 con 0x7fae900af4d0 2023-11-24T06:27:31.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae9752e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae900af0b0 con 0x7fae9006c710 2023-11-24T06:27:31.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae94ac9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fae9006c710 0x7fae900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae952ca700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fae9015e080 0x7fae9006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57398/0 (socket says 172.21.15.114:57398) 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae952ca700 1 -- 172.21.15.114:0/448377787 learned_addr learned my addr 172.21.15.114:0/448377787 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.529+0000 7fae94ac9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fae9006c710 0x7fae900b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41692/0 (socket says 172.21.15.114:41692) 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae94ac9700 1 -- 172.21.15.114:0/448377787 >> v1:172.21.15.172:6789/0 conn(0x7fae900af4d0 legacy=0x7fae9006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae94ac9700 1 -- 172.21.15.114:0/448377787 >> v1:172.21.15.114:6789/0 conn(0x7fae9015e080 legacy=0x7fae9006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae94ac9700 1 -- 172.21.15.114:0/448377787 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900ae9f0 con 0x7fae9006c710 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae94ac9700 1 --2- 172.21.15.114:0/448377787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9006c710 0x7fae900b5050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009840 tx=0x7fae8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca6027f9b70e4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae87fff700 1 -- 172.21.15.114:0/448377787 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004230 con 0x7fae9006c710 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae87fff700 1 -- 172.21.15.114:0/448377787 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c004390 con 0x7fae9006c710 2023-11-24T06:27:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae87fff700 1 -- 172.21.15.114:0/448377787 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0036a0 con 0x7fae9006c710 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 -- 172.21.15.114:0/448377787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9006c710 msgr2=0x7fae900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 --2- 172.21.15.114:0/448377787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9006c710 0x7fae900b5050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009840 tx=0x7fae8c009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 -- 172.21.15.114:0/448377787 shutdown_connections 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 --2- 172.21.15.114:0/448377787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9006c710 0x7fae900b5050 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 -- 172.21.15.114:0/448377787 >> 172.21.15.114:0/448377787 conn(0x7fae90071b90 msgr2=0x7fae90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.530+0000 7fae9752e700 1 -- 172.21.15.114:0/448377787 shutdown_connections 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.531+0000 7fae9752e700 1 -- 172.21.15.114:0/448377787 wait complete. 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 Processor -- start 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 -- start start 2023-11-24T06:27:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae9006c710 0x7fae901494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae900af4d0 0x7fae900cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90160ea0 con 0x7fae9015e080 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae90154500 con 0x7fae900af4d0 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae9752e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae901608e0 con 0x7fae9006c710 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae95acb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae95acb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41700/0 (socket says 172.21.15.114:41700) 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.532+0000 7fae95acb700 1 -- 172.21.15.114:0/2366670664 learned_addr learned my addr 172.21.15.114:0/2366670664 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae94ac9700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae900af4d0 0x7fae900cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 -- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae9006c710 msgr2=0x7fae901494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae952ca700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae9006c710 0x7fae901494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae9006c710 0x7fae901494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 -- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae900af4d0 msgr2=0x7fae900cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae900af4d0 0x7fae900cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 -- 172.21.15.114:0/2366670664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0094a0 con 0x7fae9015e080 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae95acb700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c7a0 tx=0x7fae8800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88003e10 con 0x7fae9015e080 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88005070 con 0x7fae9015e080 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae90160bc0 con 0x7fae9015e080 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88005420 con 0x7fae9015e080 2023-11-24T06:27:31.539 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.533+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae90149a00 con 0x7fae9015e080 2023-11-24T06:27:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.534+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae900753f0 con 0x7fae9015e080 2023-11-24T06:27:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.535+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fae88005580 con 0x7fae9015e080 2023-11-24T06:27:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.535+0000 7fae85ffb700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 0x7fae7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.535+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fae88089060 con 0x7fae9015e080 2023-11-24T06:27:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.535+0000 7fae952ca700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 0x7fae7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.541 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.535+0000 7fae952ca700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 0x7fae7406d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f60 tx=0x7fae80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.543 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.539+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae88053e50 con 0x7fae9015e080 2023-11-24T06:27:31.734 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.731+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae90153f80 con 0x7fae9015e080 2023-11-24T06:27:31.735 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.732+0000 7fae85ffb700 1 -- 172.21.15.114:0/2366670664 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fae90153f80 con 0x7fae9015e080 2023-11-24T06:27:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:31.736 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:25:59.863101+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:25:48.353169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:25:48.873132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:31.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.734+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 msgr2=0x7fae7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.734+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 0x7fae7406d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f60 tx=0x7fae80009040 comp rx=0 tx=0).stop 2023-11-24T06:27:31.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.734+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 msgr2=0x7fae900cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.734+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c7a0 tx=0x7fae8800cb60 comp rx=0 tx=0).stop 2023-11-24T06:27:31.738 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 shutdown_connections 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fae7406b080 0x7fae7406d540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae9015e080 0x7fae900cf540 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae9006c710 0x7fae901494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 --2- 172.21.15.114:0/2366670664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae900af4d0 0x7fae900cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 >> 172.21.15.114:0/2366670664 conn(0x7fae90071b90 msgr2=0x7fae9006e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 shutdown_connections 2023-11-24T06:27:31.739 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.735+0000 7fae9752e700 1 -- 172.21.15.114:0/2366670664 wait complete. 2023-11-24T06:27:31.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:27:31.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.868+0000 7fef01f43700 1 Processor -- start 2023-11-24T06:27:31.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7fef01f43700 1 -- start start 2023-11-24T06:27:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7fef01f43700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feefc0b0f80 0x7feefc15a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7fef01f43700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0b2bd0 con 0x7feefc0b0ba0 2023-11-24T06:27:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7fef01f43700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0b1fc0 con 0x7feefc0cd770 2023-11-24T06:27:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7fef01f43700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0b22a0 con 0x7feefc0b0f80 2023-11-24T06:27:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7feefb7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7feefc0b0ba0 0x7feefc0af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57412/0 (socket says 172.21.15.114:57412) 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.869+0000 7feefb7fe700 1 -- 172.21.15.114:0/4251985440 learned_addr learned my addr 172.21.15.114:0/4251985440 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216234541 0 0) 0x7feefc0b2bd0 con 0x7feefc0b0ba0 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee80035b0 con 0x7feefc0b0ba0 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefaffd700 1 --2- 172.21.15.114:0/4251985440 >> v2:172.21.15.172:3300/0 conn(0x7feefc0b0f80 0x7feefc15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3835248337 0 0) 0x7feee80035b0 con 0x7feefc0b0ba0 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 >> v2:172.21.15.172:3300/0 conn(0x7feefc0b0f80 msgr2=0x7feefc15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 --2- 172.21.15.114:0/4251985440 >> v2:172.21.15.172:3300/0 conn(0x7feefc0b0f80 0x7feefc15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 >> v1:172.21.15.172:6789/0 conn(0x7feefc0cd770 legacy=0x7feefc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0b2580 con 0x7feefc0b0ba0 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7feeec001670 con 0x7feefc0b0ba0 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 >> v1:172.21.15.114:6789/0 conn(0x7feefc0b0ba0 legacy=0x7feefc0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a00 0x7feee8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8004850 0x7feee8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 0x7feee800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b22a0 con 0x7feee8005210 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee800c160 con 0x7feee8003a00 2023-11-24T06:27:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee800d430 con 0x7feee8004850 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 0x7feee800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefb7fe700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a00 0x7feee8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefaffd700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8004850 0x7feee8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 -- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8004850 msgr2=0x7feee8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8004850 0x7feee8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 -- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a00 msgr2=0x7feee8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee8003a00 0x7feee8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 -- 172.21.15.114:0/4251985440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80035b0 con 0x7feee8005210 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.870+0000 7feefbfff700 1 -- 172.21.15.114:0/4251985440 reap_dead start 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7feefbfff700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 0x7feee800ba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feef400c190 tx=0x7feee8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499b32afb1d11cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4001db0 con 0x7feee8005210 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400ccd0 con 0x7feee8005210 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7feefa7fc700 1 -- 172.21.15.114:0/4251985440 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4003c40 con 0x7feee8005210 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 -- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 msgr2=0x7feee800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 0x7feee800ba10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feef400c190 tx=0x7feee8005880 comp rx=0 tx=0).stop 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 -- 172.21.15.114:0/4251985440 shutdown_connections 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 --2- 172.21.15.114:0/4251985440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feee8005210 0x7feee800ba10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 -- 172.21.15.114:0/4251985440 >> 172.21.15.114:0/4251985440 conn(0x7feefc0b57b0 msgr2=0x7feefc0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 -- 172.21.15.114:0/4251985440 shutdown_connections 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.871+0000 7fef01f43700 1 -- 172.21.15.114:0/4251985440 wait complete. 2023-11-24T06:27:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 Processor -- start 2023-11-24T06:27:31.881 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 -- start start 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 0x7feefc0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1569d0 0x7feefc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc151610 con 0x7feefc0ad270 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1f2dc0 con 0x7feefc1569d0 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.873+0000 7fef01f43700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0ad7e0 con 0x7feee8005210 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41718/0 (socket says 172.21.15.114:41718) 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 -- 172.21.15.114:0/2298994777 learned_addr learned my addr 172.21.15.114:0/2298994777 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefbfff700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1569d0 0x7feefc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefb7fe700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 0x7feefc0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 -- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 msgr2=0x7feefc0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 0x7feefc0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 -- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1569d0 msgr2=0x7feefc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1569d0 0x7feefc0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 -- 172.21.15.114:0/2298994777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0adac0 con 0x7feefc0ad270 2023-11-24T06:27:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefb7fe700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 0x7feefc0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feefaffd700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feee400c8d0 tx=0x7feee400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40137b0 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4013910 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0adda0 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4013ca0 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.874+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1f2fd0 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.875+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0b1d70 con 0x7feefc0ad270 2023-11-24T06:27:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.876+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7feee4014040 con 0x7feefc0ad270 2023-11-24T06:27:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.876+0000 7feef3fff700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 0x7feedc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.876+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feee408bb20 con 0x7feefc0ad270 2023-11-24T06:27:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.876+0000 7feefb7fe700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 0x7feedc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.876+0000 7feefb7fe700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 0x7feedc06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feeec007dd0 tx=0x7feeec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:31.881+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feee4056800 con 0x7feefc0ad270 2023-11-24T06:27:32.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.119+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feefc067f60 con 0x7feefc0ad270 2023-11-24T06:27:32.123 INFO:tasks.ceph.mon.a.smithi114.stderr:2023-11-24T06:27:32.119+0000 7f8fd8393700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:27:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.123+0000 7feef3fff700 1 -- 172.21.15.114:0/2298994777 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7feee4025090 con 0x7feefc0ad270 2023-11-24T06:27:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.126+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 msgr2=0x7feedc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.126+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 0x7feedc06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feeec007dd0 tx=0x7feeec009040 comp rx=0 tx=0).stop 2023-11-24T06:27:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.126+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 msgr2=0x7feefc1510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.126+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feee400c8d0 tx=0x7feee400cbe0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.126+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 shutdown_connections 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7feedc06b080 0x7feedc06d540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feefc1569d0 0x7feefc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feee8005210 0x7feefc0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 --2- 172.21.15.114:0/2298994777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feefc0ad270 0x7feefc1510d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 >> 172.21.15.114:0/2298994777 conn(0x7feefc0b57b0 msgr2=0x7feefc15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 shutdown_connections 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.127+0000 7fef01f43700 1 -- 172.21.15.114:0/2298994777 wait complete. 2023-11-24T06:27:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:27:32.143 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:27:32.167 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2023-11-24T06:27:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.262+0000 7f5de201c700 1 Processor -- start 2023-11-24T06:27:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.262+0000 7f5de201c700 1 -- start start 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.263+0000 7f5de201c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.263+0000 7f5de201c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ddc0746a0 0x7f5ddc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.263+0000 7f5de201c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc0b3c60 con 0x7f5ddc0afc90 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.263+0000 7f5de201c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc0b4500 con 0x7f5ddc0746a0 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.263+0000 7f5de201c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0b58c0 con 0x7f5ddc15e160 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ddc0746a0 0x7f5ddc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddaffd700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5ddc0afc90 0x7f5ddc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:51048/0 (socket says 172.21.15.114:51048) 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddaffd700 1 -- 172.21.15.114:0/1682456724 learned_addr learned my addr 172.21.15.114:0/1682456724 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 -- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 msgr2=0x7f5ddc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 --2- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 -- 172.21.15.114:0/1682456724 >> v1:172.21.15.172:6789/0 conn(0x7f5ddc0afc90 legacy=0x7f5ddc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5dda7fc700 1 -- 172.21.15.114:0/1682456724 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004285203 0 0) 0x7f5ddc0b3c60 con 0x7f5ddc0afc90 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 -- 172.21.15.114:0/1682456724 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc06bdc0 con 0x7f5ddc0746a0 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5ddbfff700 1 --2- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc06d930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009d90 tx=0x7f5dd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec809e282e493db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5dda7fc700 1 -- 172.21.15.114:0/1682456724 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400e070 con 0x7f5ddc0746a0 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5dda7fc700 1 -- 172.21.15.114:0/1682456724 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400bb20 con 0x7f5ddc0746a0 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.264+0000 7f5dda7fc700 1 -- 172.21.15.114:0/1682456724 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4012430 con 0x7f5ddc0746a0 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 -- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0746a0 msgr2=0x7f5ddc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 --2- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc06d930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009d90 tx=0x7f5dd400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:32.271 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 -- 172.21.15.114:0/1682456724 shutdown_connections 2023-11-24T06:27:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 --2- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc06d930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 --2- 172.21.15.114:0/1682456724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 -- 172.21.15.114:0/1682456724 >> 172.21.15.114:0/1682456724 conn(0x7f5ddc0d0bf0 msgr2=0x7f5ddc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 -- 172.21.15.114:0/1682456724 shutdown_connections 2023-11-24T06:27:32.272 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.265+0000 7f5de201c700 1 -- 172.21.15.114:0/1682456724 wait complete. 2023-11-24T06:27:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.266+0000 7f5de201c700 1 Processor -- start 2023-11-24T06:27:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.266+0000 7f5de201c700 1 -- start start 2023-11-24T06:27:32.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 0x7f5ddc158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc160270 con 0x7f5ddc0746a0 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc15f9d0 con 0x7f5ddc0afc90 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5de201c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc15fcb0 con 0x7f5ddc15e160 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41748/0 (socket says 172.21.15.114:41748) 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 -- 172.21.15.114:0/1038497674 learned_addr learned my addr 172.21.15.114:0/1038497674 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddaffd700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 0x7f5ddc158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddbfff700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 -- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 msgr2=0x7f5ddc0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 -- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 msgr2=0x7f5ddc158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 0x7f5ddc158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddb7fe700 1 -- 172.21.15.114:0/1038497674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4009a70 con 0x7f5ddc0746a0 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.267+0000 7f5ddaffd700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 0x7f5ddc158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:32.277 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5ddb7fe700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400e870 tx=0x7f5dc400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4009590 con 0x7f5ddc0746a0 2023-11-24T06:27:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc400ee70 con 0x7f5ddc0746a0 2023-11-24T06:27:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc15ff90 con 0x7f5ddc0746a0 2023-11-24T06:27:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400ca50 con 0x7f5ddc0746a0 2023-11-24T06:27:32.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.268+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc158dd0 con 0x7f5ddc0746a0 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.269+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc0c5bd0 con 0x7f5ddc0746a0 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.269+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5dc400cbb0 con 0x7f5ddc0746a0 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.269+0000 7f5dd3fff700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 0x7f5dbc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.270+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5dc4089880 con 0x7f5ddc0746a0 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.270+0000 7f5ddaffd700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 0x7f5dbc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.270+0000 7f5ddaffd700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 0x7f5dbc06d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc15be40 tx=0x7f5dcc007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.280 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.274+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ddc0c5bd0 con 0x7f5ddc0746a0 2023-11-24T06:27:32.507 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:32.504+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ddc0c5bd0 con 0x7f5ddc0746a0 2023-11-24T06:27:33.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.124+0000 7f5dd3fff700 1 -- 172.21.15.114:0/1038497674 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ddc0c5bd0 con 0x7f5ddc0746a0 2023-11-24T06:27:33.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 msgr2=0x7f5dbc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 0x7f5dbc06d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc15be40 tx=0x7f5dcc007ca0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 msgr2=0x7f5ddc15f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400e870 tx=0x7f5dc400ec30 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 shutdown_connections 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5dbc06b080 0x7f5dbc06d540 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ddc15e160 0x7f5ddc0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ddc0746a0 0x7f5ddc15f180 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 --2- 172.21.15.114:0/1038497674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ddc0afc90 0x7f5ddc158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 >> 172.21.15.114:0/1038497674 conn(0x7f5ddc0d0bf0 msgr2=0x7f5ddc15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 shutdown_connections 2023-11-24T06:27:33.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.127+0000 7f5de201c700 1 -- 172.21.15.114:0/1038497674 wait complete. 2023-11-24T06:27:33.144 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:27:33.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.306+0000 7f509add1700 1 Processor -- start 2023-11-24T06:27:33.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.306+0000 7f509add1700 1 -- start start 2023-11-24T06:27:33.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509add1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 0x7f50940b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509add1700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 0x7f50940cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509add1700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940b1ba0 con 0x7f50940cf910 2023-11-24T06:27:33.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509add1700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5094161c80 con 0x7f50940b1720 2023-11-24T06:27:33.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509add1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5094161f40 con 0x7f509415e020 2023-11-24T06:27:33.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f5093fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 0x7f50940cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f5098b6d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 0x7f50940b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509936e700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f50940cf910 0x7f509415e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:51064/0 (socket says 172.21.15.114:51064) 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.307+0000 7f509936e700 1 -- 172.21.15.114:0/2770949190 learned_addr learned my addr 172.21.15.114:0/2770949190 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f50937fe700 1 -- 172.21.15.114:0/2770949190 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212972057 0 0) 0x7f50940b1ba0 con 0x7f50940cf910 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f50937fe700 1 -- 172.21.15.114:0/2770949190 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50840035b0 con 0x7f50940cf910 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5098b6d700 1 -- 172.21.15.114:0/2770949190 >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 msgr2=0x7f50940cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5098b6d700 1 --2- 172.21.15.114:0/2770949190 >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 0x7f50940cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5098b6d700 1 -- 172.21.15.114:0/2770949190 >> v1:172.21.15.172:6789/0 conn(0x7f50940cf910 legacy=0x7f509415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5098b6d700 1 -- 172.21.15.114:0/2770949190 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094162220 con 0x7f509415e020 2023-11-24T06:27:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5093fff700 1 --2- 172.21.15.114:0/2770949190 >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 0x7f50940cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f5098b6d700 1 --2- 172.21.15.114:0/2770949190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 0x7f50940b11e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7c5de2a00221be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f50937fe700 1 -- 172.21.15.114:0/2770949190 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011830 con 0x7f509415e020 2023-11-24T06:27:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f50937fe700 1 -- 172.21.15.114:0/2770949190 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088011990 con 0x7f509415e020 2023-11-24T06:27:33.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.308+0000 7f50937fe700 1 -- 172.21.15.114:0/2770949190 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011d20 con 0x7f509415e020 2023-11-24T06:27:33.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 -- 172.21.15.114:0/2770949190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 msgr2=0x7f50940b11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 --2- 172.21.15.114:0/2770949190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 0x7f50940b11e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:33.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 -- 172.21.15.114:0/2770949190 shutdown_connections 2023-11-24T06:27:33.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 --2- 172.21.15.114:0/2770949190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f509415e020 0x7f50940b11e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 --2- 172.21.15.114:0/2770949190 >> v2:172.21.15.172:3300/0 conn(0x7f50940b1720 0x7f50940cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 -- 172.21.15.114:0/2770949190 >> 172.21.15.114:0/2770949190 conn(0x7f5094072710 msgr2=0x7f5094072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 -- 172.21.15.114:0/2770949190 shutdown_connections 2023-11-24T06:27:33.318 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.309+0000 7f509add1700 1 -- 172.21.15.114:0/2770949190 wait complete. 2023-11-24T06:27:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.310+0000 7f509add1700 1 Processor -- start 2023-11-24T06:27:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.310+0000 7f509add1700 1 -- start start 2023-11-24T06:27:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50940b1720 0x7f5094155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50940cf910 0x7f5094153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5094153cb0 con 0x7f509415e020 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940c64a0 con 0x7f50940cf910 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509add1700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5094153f10 con 0x7f50940b1720 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41768/0 (socket says 172.21.15.114:41768) 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 -- 172.21.15.114:0/2092249172 learned_addr learned my addr 172.21.15.114:0/2092249172 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f5098b6d700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50940b1720 0x7f5094155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f5093fff700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50940cf910 0x7f5094153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 -- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50940b1720 msgr2=0x7f5094155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50940b1720 0x7f5094155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.323 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 -- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50940cf910 msgr2=0x7f5094153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50940cf910 0x7f5094153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 -- 172.21.15.114:0/2092249172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800094a0 con 0x7f509415e020 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f509936e700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cb40 tx=0x7f508c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.311+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00ab50 con 0x7f509415e020 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.312+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00acb0 con 0x7f509415e020 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.312+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50880094a0 con 0x7f509415e020 2023-11-24T06:27:33.324 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.312+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c012620 con 0x7f509415e020 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.312+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50941544d0 con 0x7f509415e020 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.313+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5094067f60 con 0x7f509415e020 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.313+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f508c01b3b0 con 0x7f509415e020 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.313+0000 7f50917fa700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 0x7f507806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.313+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f508c0897c0 con 0x7f509415e020 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.313+0000 7f5098b6d700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 0x7f507806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.314+0000 7f5098b6d700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 0x7f507806d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5088000c00 tx=0x7f5088008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.318+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f508c054560 con 0x7f509415e020 2023-11-24T06:27:33.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:33.510+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50940732c0 con 0x7f509415e020 2023-11-24T06:27:34.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.128+0000 7f50917fa700 1 -- 172.21.15.114:0/2092249172 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f508c057b80 con 0x7f509415e020 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 msgr2=0x7f507806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 0x7f507806d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5088000c00 tx=0x7f5088008030 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 msgr2=0x7f5094153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cb40 tx=0x7f508c00cf00 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 shutdown_connections 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f507806b080 0x7f507806d540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f509415e020 0x7f5094153770 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50940b1720 0x7f5094155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 --2- 172.21.15.114:0/2092249172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50940cf910 0x7f5094153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 >> 172.21.15.114:0/2092249172 conn(0x7f5094072710 msgr2=0x7f509406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 shutdown_connections 2023-11-24T06:27:34.135 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.131+0000 7f509add1700 1 -- 172.21.15.114:0/2092249172 wait complete. 2023-11-24T06:27:34.136 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:27:34.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:27:34.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.279+0000 7f6d7000c700 1 Processor -- start 2023-11-24T06:27:34.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 -- start start 2023-11-24T06:27:34.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d6806f480 0x7f6d6806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d6806e830 0x7f6d6806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6806b200 con 0x7f6d68144600 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6806aff0 con 0x7f6d6806e830 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.280+0000 7f6d7000c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6806f1e0 con 0x7f6d6806f480 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d6806e830 0x7f6d6806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d6806e830 0x7f6d6806eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41776/0 (socket says 172.21.15.114:41776) 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 -- 172.21.15.114:0/536231071 learned_addr learned my addr 172.21.15.114:0/536231071 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 -- 172.21.15.114:0/536231071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d6806f480 msgr2=0x7f6d6806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 --2- 172.21.15.114:0/536231071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d6806f480 0x7f6d6806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 -- 172.21.15.114:0/536231071 >> v1:172.21.15.172:6789/0 conn(0x7f6d68144600 legacy=0x7f6d680b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 -- 172.21.15.114:0/536231071 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6806b4e0 con 0x7f6d6806e830 2023-11-24T06:27:34.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6e5a9700 1 --2- 172.21.15.114:0/536231071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806e830 0x7f6d6806eca0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009e50 tx=0x7f6d6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53077248cfa3bfde server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6cda6700 1 -- 172.21.15.114:0/536231071 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000e070 con 0x7f6d6806e830 2023-11-24T06:27:34.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6cda6700 1 -- 172.21.15.114:0/536231071 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60003a20 con 0x7f6d6806e830 2023-11-24T06:27:34.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.281+0000 7f6d6cda6700 1 -- 172.21.15.114:0/536231071 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60003dd0 con 0x7f6d6806e830 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 -- 172.21.15.114:0/536231071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806e830 msgr2=0x7f6d6806eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 --2- 172.21.15.114:0/536231071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806e830 0x7f6d6806eca0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009e50 tx=0x7f6d6000b330 comp rx=0 tx=0).stop 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 -- 172.21.15.114:0/536231071 shutdown_connections 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 --2- 172.21.15.114:0/536231071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806e830 0x7f6d6806eca0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 --2- 172.21.15.114:0/536231071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d6806f480 0x7f6d6806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 -- 172.21.15.114:0/536231071 >> 172.21.15.114:0/536231071 conn(0x7f6d680d0b40 msgr2=0x7f6d680d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 -- 172.21.15.114:0/536231071 shutdown_connections 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.282+0000 7f6d7000c700 1 -- 172.21.15.114:0/536231071 wait complete. 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.283+0000 7f6d7000c700 1 Processor -- start 2023-11-24T06:27:34.289 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.283+0000 7f6d7000c700 1 -- start start 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6806e830 0x7f6d6813ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d68144600 0x7f6d6815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6815c410 con 0x7f6d6806f480 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d681f7e40 con 0x7f6d6806e830 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d7000c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6813f160 con 0x7f6d68144600 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41780/0 (socket says 172.21.15.114:41780) 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 -- 172.21.15.114:0/1899890559 learned_addr learned my addr 172.21.15.114:0/1899890559 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6dda8700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6806e830 0x7f6d6813ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6e5a9700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d68144600 0x7f6d6815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 -- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d68144600 msgr2=0x7f6d6815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d68144600 0x7f6d6815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 -- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6806e830 msgr2=0x7f6d6813ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6806e830 0x7f6d6813ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 -- 172.21.15.114:0/1899890559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60009b30 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d6d5a7700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400b530 tx=0x7f6d6400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.284+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64011830 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.285+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64011990 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.285+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d681f72c0 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.285+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64011d40 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.285+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d681f7880 con 0x7f6d6806f480 2023-11-24T06:27:34.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.286+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d68067f60 con 0x7f6d6806f480 2023-11-24T06:27:34.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.286+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6d6401a400 con 0x7f6d6806f480 2023-11-24T06:27:34.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.286+0000 7f6d5e7fc700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 0x7f6d4c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.286+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f6d64089720 con 0x7f6d6806f480 2023-11-24T06:27:34.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.286+0000 7f6d6dda8700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 0x7f6d4c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.287+0000 7f6d6dda8700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 0x7f6d4c06d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d6815ac20 tx=0x7f6d580090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.291+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d64054730 con 0x7f6d6806f480 2023-11-24T06:27:34.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:34.483+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d6813f390 con 0x7f6d6806f480 2023-11-24T06:27:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.135+0000 7f6d5e7fc700 1 -- 172.21.15.114:0/1899890559 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6d640150d0 con 0x7f6d6806f480 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 msgr2=0x7f6d4c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 0x7f6d4c06d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d6815ac20 tx=0x7f6d580090e0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 msgr2=0x7f6d6815e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400b530 tx=0x7f6d6400b840 comp rx=0 tx=0).stop 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 shutdown_connections 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d4c06b080 0x7f6d4c06d540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d68144600 0x7f6d6815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6806e830 0x7f6d6813ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 --2- 172.21.15.114:0/1899890559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d6806f480 0x7f6d6815e7e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 >> 172.21.15.114:0/1899890559 conn(0x7f6d680d0b40 msgr2=0x7f6d680b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 shutdown_connections 2023-11-24T06:27:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.139+0000 7f6d7000c700 1 -- 172.21.15.114:0/1899890559 wait complete. 2023-11-24T06:27:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2023-11-24T06:27:35.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:27:35.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.279+0000 7f8536da9700 1 Processor -- start 2023-11-24T06:27:35.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.279+0000 7f8536da9700 1 -- start start 2023-11-24T06:27:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f8536da9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f85300b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f8536da9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f85300afc90 0x7f8530073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f8536da9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85300b3c60 con 0x7f85300744d0 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f8536da9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85300b4500 con 0x7f85300afc90 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f8536da9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8530074960 con 0x7f853015e160 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f85300afc90 0x7f8530073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f85300afc90 0x7f8530073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41786/0 (socket says 172.21.15.114:41786) 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 -- 172.21.15.114:0/2238490774 learned_addr learned my addr 172.21.15.114:0/2238490774 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 -- 172.21.15.114:0/2238490774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 msgr2=0x7f85300b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 --2- 172.21.15.114:0/2238490774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f85300b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 -- 172.21.15.114:0/2238490774 >> v1:172.21.15.172:6789/0 conn(0x7f85300744d0 legacy=0x7f853006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 -- 172.21.15.114:0/2238490774 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300b58c0 con 0x7f85300afc90 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.280+0000 7f852ffff700 1 --2- 172.21.15.114:0/2238490774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300afc90 0x7f8530073f90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f851c0098f0 tx=0x7f851c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e795c7d916938f69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f852f7fe700 1 -- 172.21.15.114:0/2238490774 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004230 con 0x7f85300afc90 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f852f7fe700 1 -- 172.21.15.114:0/2238490774 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c004390 con 0x7f85300afc90 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f852f7fe700 1 -- 172.21.15.114:0/2238490774 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0036a0 con 0x7f85300afc90 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 -- 172.21.15.114:0/2238490774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300afc90 msgr2=0x7f8530073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 --2- 172.21.15.114:0/2238490774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300afc90 0x7f8530073f90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f851c0098f0 tx=0x7f851c009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:35.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 -- 172.21.15.114:0/2238490774 shutdown_connections 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 --2- 172.21.15.114:0/2238490774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f85300b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 --2- 172.21.15.114:0/2238490774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300afc90 0x7f8530073f90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 -- 172.21.15.114:0/2238490774 >> 172.21.15.114:0/2238490774 conn(0x7f85300d0bf0 msgr2=0x7f85300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 -- 172.21.15.114:0/2238490774 shutdown_connections 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.281+0000 7f8536da9700 1 -- 172.21.15.114:0/2238490774 wait complete. 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.282+0000 7f8536da9700 1 Processor -- start 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.282+0000 7f8536da9700 1 -- start start 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 0x7f8530148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f8530148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853015e5e0 con 0x7f85300744d0 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853015e880 con 0x7f85300afc90 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8536da9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853015eb60 con 0x7f853015e160 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8534b45700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8534b45700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41792/0 (socket says 172.21.15.114:41792) 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8534b45700 1 -- 172.21.15.114:0/445003202 learned_addr learned my addr 172.21.15.114:0/445003202 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:35.294 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f852ffff700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 0x7f8530148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.283+0000 7f8535346700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f8530148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 -- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 msgr2=0x7f8530148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f8530148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 -- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 msgr2=0x7f8530148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 0x7f8530148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 -- 172.21.15.114:0/445003202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009550 con 0x7f85300744d0 2023-11-24T06:27:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f852ffff700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 0x7f8530148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8534b45700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f852400b4a0 tx=0x7f852400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524011830 con 0x7f85300744d0 2023-11-24T06:27:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524011990 con 0x7f85300744d0 2023-11-24T06:27:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524011d40 con 0x7f85300744d0 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853015ee40 con 0x7f85300744d0 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.284+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853006f130 con 0x7f85300744d0 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.285+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85300b54a0 con 0x7f85300744d0 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.285+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f852401a400 con 0x7f85300744d0 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.286+0000 7f852d7fa700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 0x7f851406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.297 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.286+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f85240895b0 con 0x7f85300744d0 2023-11-24T06:27:35.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.286+0000 7f852ffff700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 0x7f851406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.298 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.286+0000 7f852ffff700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 0x7f851406d570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f851c009130 tx=0x7f851c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.290+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85240546f0 con 0x7f85300744d0 2023-11-24T06:27:35.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.482+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8530068fb0 con 0x7f85300744d0 2023-11-24T06:27:35.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.485+0000 7f852d7fa700 1 -- 172.21.15.114:0/445003202 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8524057d10 con 0x7f85300744d0 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 msgr2=0x7f851406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 0x7f851406d570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f851c009130 tx=0x7f851c008030 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 msgr2=0x7f853006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f852400b4a0 tx=0x7f852400b860 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 shutdown_connections 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f851406b0b0 0x7f851406d570 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f853015e160 0x7f8530148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85300744d0 0x7f853006ebf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 --2- 172.21.15.114:0/445003202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85300afc90 0x7f8530148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 >> 172.21.15.114:0/445003202 conn(0x7f85300d0bf0 msgr2=0x7f85300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 shutdown_connections 2023-11-24T06:27:35.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.487+0000 7f8536da9700 1 -- 172.21.15.114:0/445003202 wait complete. 2023-11-24T06:27:35.503 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-24T06:27:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.620+0000 7f9a1e8de700 1 Processor -- start 2023-11-24T06:27:35.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.620+0000 7f9a1e8de700 1 -- start start 2023-11-24T06:27:35.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a1e8de700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a1806c710 0x7f9a180b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a1e8de700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180ae430 con 0x7f9a1815e080 2023-11-24T06:27:35.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a1e8de700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180ae710 con 0x7f9a180af4d0 2023-11-24T06:27:35.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a1e8de700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180af0b0 con 0x7f9a1806c710 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a17fff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9a1815e080 0x7f9a1806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57508/0 (socket says 172.21.15.114:57508) 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a17fff700 1 -- 172.21.15.114:0/716608177 learned_addr learned my addr 172.21.15.114:0/716608177 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379664195 0 0) 0x7f9a180ae430 con 0x7f9a1815e080 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a000035b0 con 0x7f9a1815e080 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a177fe700 1 --2- 172.21.15.114:0/716608177 >> v2:172.21.15.172:3300/0 conn(0x7f9a1806c710 0x7f9a180b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054472900 0 0) 0x7f9a000035b0 con 0x7f9a1815e080 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 >> v2:172.21.15.172:3300/0 conn(0x7f9a1806c710 msgr2=0x7f9a180b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 --2- 172.21.15.114:0/716608177 >> v2:172.21.15.172:3300/0 conn(0x7f9a1806c710 0x7f9a180b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 >> v1:172.21.15.172:6789/0 conn(0x7f9a180af4d0 legacy=0x7f9a1806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180ae9f0 con 0x7f9a1815e080 2023-11-24T06:27:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.621+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f9a0c001670 con 0x7f9a1815e080 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 >> v1:172.21.15.114:6789/0 conn(0x7f9a1815e080 legacy=0x7f9a1806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a00004850 0x7f9a00004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a00005210 0x7f9a0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180af0b0 con 0x7f9a00003a00 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000c0b0 con 0x7f9a00004850 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000c270 con 0x7f9a00005210 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a177fe700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a00004850 0x7f9a00004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a1ce7b700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a00005210 0x7f9a0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 -- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a00005210 msgr2=0x7f9a0000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a00005210 0x7f9a0000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 -- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a00004850 msgr2=0x7f9a00004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a00004850 0x7f9a00004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 -- 172.21.15.114:0/716608177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000035b0 con 0x7f9a00003a00 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a1ce7b700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a00005210 0x7f9a0000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a1ce7b700 1 -- 172.21.15.114:0/716608177 reap_dead start 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a17fff700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a00003e20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c240 tx=0x7f9a00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313413c5df2cbf01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c007c10 con 0x7f9a00003a00 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00cde0 con 0x7f9a00003a00 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.622+0000 7f9a16ffd700 1 -- 172.21.15.114:0/716608177 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c005ba0 con 0x7f9a00003a00 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 -- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 msgr2=0x7f9a00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a00003e20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c240 tx=0x7f9a00005880 comp rx=0 tx=0).stop 2023-11-24T06:27:35.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 -- 172.21.15.114:0/716608177 shutdown_connections 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/716608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a00003e20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 -- 172.21.15.114:0/716608177 >> 172.21.15.114:0/716608177 conn(0x7f9a18071b90 msgr2=0x7f9a18071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 -- 172.21.15.114:0/716608177 shutdown_connections 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.623+0000 7f9a1e8de700 1 -- 172.21.15.114:0/716608177 wait complete. 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.624+0000 7f9a1e8de700 1 Processor -- start 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.624+0000 7f9a1e8de700 1 -- start start 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a180cbda0 0x7f9a1814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a180c9270 0x7f9a180c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180c9c40 con 0x7f9a00003a00 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180ce6e0 con 0x7f9a180c9270 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a1e8de700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180cc310 con 0x7f9a180cbda0 2023-11-24T06:27:35.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41812/0 (socket says 172.21.15.114:41812) 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 -- 172.21.15.114:0/1389016499 learned_addr learned my addr 172.21.15.114:0/1389016499 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 -- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a180cbda0 msgr2=0x7f9a1814c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a180cbda0 0x7f9a1814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 -- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a180c9270 msgr2=0x7f9a180c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a180c9270 0x7f9a180c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 -- 172.21.15.114:0/1389016499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180cc5f0 con 0x7f9a00003a00 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.625+0000 7f9a17fff700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c080 tx=0x7f9a0c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.626+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0070d0 con 0x7f9a00003a00 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.626+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c007250 con 0x7f9a00003a00 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.626+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180cc8d0 con 0x7f9a00003a00 2023-11-24T06:27:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.626+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c007600 con 0x7f9a00003a00 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.626+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a180c6e30 con 0x7f9a00003a00 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.627+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a180753f0 con 0x7f9a00003a00 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.627+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9a0c027450 con 0x7f9a00003a00 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.627+0000 7f9a14ff9700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 0x7f9a0406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.628+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a0c08ab80 con 0x7f9a00003a00 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.628+0000 7f9a177fe700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 0x7f9a0406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.628+0000 7f9a177fe700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 0x7f9a0406d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009950 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.632+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a0c055cc0 con 0x7f9a00003a00 2023-11-24T06:27:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:35.911+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f9a18067f60 con 0x7f9a00003a00 2023-11-24T06:27:36.140 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.136+0000 7f9a14ff9700 1 -- 172.21.15.114:0/1389016499 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v185) v1 ==== 140+0+0 (secure 0 0 0) 0x7f9a0c0592e0 con 0x7f9a00003a00 2023-11-24T06:27:36.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 msgr2=0x7f9a0406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 0x7f9a0406d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009950 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2023-11-24T06:27:36.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 msgr2=0x7f9a180c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c080 tx=0x7f9a0c001a00 comp rx=0 tx=0).stop 2023-11-24T06:27:36.144 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 shutdown_connections 2023-11-24T06:27:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9a0406b080 0x7f9a0406d540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a180c9270 0x7f9a180c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a00003a00 0x7f9a180c68f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 --2- 172.21.15.114:0/1389016499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a180cbda0 0x7f9a1814c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 >> 172.21.15.114:0/1389016499 conn(0x7f9a18071b90 msgr2=0x7f9a1806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 shutdown_connections 2023-11-24T06:27:36.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.140+0000 7f9a1e8de700 1 -- 172.21.15.114:0/1389016499 wait complete. 2023-11-24T06:27:36.156 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-24T06:27:36.156 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-24T06:27:36.156 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:27:36.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.273+0000 7f3708c8f700 1 Processor -- start 2023-11-24T06:27:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.273+0000 7f3708c8f700 1 -- start start 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3708c8f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3704145510 0x7f3704073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3708c8f700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f37040ceb70 0x7f370406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3708c8f700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370415fb30 con 0x7f37040744d0 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3708c8f700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3704074950 con 0x7f37040ceb70 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3708c8f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040cefa0 con 0x7f3704145510 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f37040ceb70 0x7f370406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3701d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f37040744d0 0x7f37040ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57530/0 (socket says 172.21.15.114:57530) 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3701d9b700 1 -- 172.21.15.114:0/4180476356 learned_addr learned my addr 172.21.15.114:0/4180476356 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 -- 172.21.15.114:0/4180476356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3704145510 msgr2=0x7f3704073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 --2- 172.21.15.114:0/4180476356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3704145510 0x7f3704073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 -- 172.21.15.114:0/4180476356 >> v1:172.21.15.114:6789/0 conn(0x7f37040744d0 legacy=0x7f37040ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 -- 172.21.15.114:0/4180476356 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370415fd80 con 0x7f37040ceb70 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f370159a700 1 -- 172.21.15.114:0/4180476356 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747412340 0 0) 0x7f370415fb30 con 0x7f37040744d0 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f3702d9d700 1 --2- 172.21.15.114:0/4180476356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37040ceb70 0x7f370406eec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009d90 tx=0x7f36fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a9fbee550ee9d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f370159a700 1 -- 172.21.15.114:0/4180476356 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00e070 con 0x7f37040ceb70 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f370159a700 1 -- 172.21.15.114:0/4180476356 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc00bbd0 con 0x7f37040ceb70 2023-11-24T06:27:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.274+0000 7f370159a700 1 -- 172.21.15.114:0/4180476356 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc012430 con 0x7f37040ceb70 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 -- 172.21.15.114:0/4180476356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37040ceb70 msgr2=0x7f370406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 --2- 172.21.15.114:0/4180476356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37040ceb70 0x7f370406eec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009d90 tx=0x7f36fc00b330 comp rx=0 tx=0).stop 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 -- 172.21.15.114:0/4180476356 shutdown_connections 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 --2- 172.21.15.114:0/4180476356 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3704145510 0x7f3704073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 --2- 172.21.15.114:0/4180476356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37040ceb70 0x7f370406eec0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 -- 172.21.15.114:0/4180476356 >> 172.21.15.114:0/4180476356 conn(0x7f3704072990 msgr2=0x7f370406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 -- 172.21.15.114:0/4180476356 shutdown_connections 2023-11-24T06:27:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.275+0000 7f3708c8f700 1 -- 172.21.15.114:0/4180476356 wait complete. 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.276+0000 7f3708c8f700 1 Processor -- start 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.276+0000 7f3708c8f700 1 -- start start 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 0x7f37041e6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 0x7f37040b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370413c000 con 0x7f3704145510 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040b50d0 con 0x7f37040744d0 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3708c8f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040b53b0 con 0x7f37040ceb70 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3702d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3702d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41830/0 (socket says 172.21.15.114:41830) 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3702d9d700 1 -- 172.21.15.114:0/3371589267 learned_addr learned my addr 172.21.15.114:0/3371589267 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f3701d9b700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 0x7f37040b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.277+0000 7f370259c700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 0x7f37041e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 -- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 msgr2=0x7f37040b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 0x7f37040b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 -- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 msgr2=0x7f37041e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 0x7f37041e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 -- 172.21.15.114:0/3371589267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009a70 con 0x7f3704145510 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3701d9b700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 0x7f37040b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f370259c700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 0x7f37041e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3702d9d700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f37040b5fb0 tx=0x7f36fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc012430 con 0x7f3704145510 2023-11-24T06:27:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc020070 con 0x7f3704145510 2023-11-24T06:27:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc004280 con 0x7f3704145510 2023-11-24T06:27:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37040b5690 con 0x7f3704145510 2023-11-24T06:27:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.278+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37040b4da0 con 0x7f3704145510 2023-11-24T06:27:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.279+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370413c2e0 con 0x7f3704145510 2023-11-24T06:27:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.279+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f36fc0043e0 con 0x7f3704145510 2023-11-24T06:27:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.280+0000 7f36f2ffd700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.280+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f36fc022070 con 0x7f3704145510 2023-11-24T06:27:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.280+0000 7f370259c700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.280+0000 7f370259c700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 0x7f36e406d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007900 tx=0x7f36f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.284+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36fc056630 con 0x7f3704145510 2023-11-24T06:27:36.477 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.474+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370413c140 con 0x7f3704145510 2023-11-24T06:27:36.478 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.475+0000 7f36f2ffd700 1 -- 172.21.15.114:0/3371589267 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13024 (secure 0 0 0) 0x7f370413c140 con 0x7f3704145510 2023-11-24T06:27:36.478 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:36.478 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":50,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:35.130753+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6837/4123842361":"2023-11-25T06:27:32.120934+0000","172.21.15.114:6835/4123842361":"2023-11-25T06:27:32.120934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:36.483 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.476+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 msgr2=0x7f36e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.476+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 0x7f36e406d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007900 tx=0x7f36f8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 msgr2=0x7f370415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f37040b5fb0 tx=0x7f36fc004ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 shutdown_connections 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704145510 0x7f370415b260 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37040744d0 0x7f37041e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 --2- 172.21.15.114:0/3371589267 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f37040ceb70 0x7f37040b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 >> 172.21.15.114:0/3371589267 conn(0x7f3704072990 msgr2=0x7f370406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 shutdown_connections 2023-11-24T06:27:36.486 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.477+0000 7f3708c8f700 1 -- 172.21.15.114:0/3371589267 wait complete. 2023-11-24T06:27:36.493 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:36.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.613+0000 7ff575c11700 1 Processor -- start 2023-11-24T06:27:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.613+0000 7ff575c11700 1 -- start start 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff575c11700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff57015a620 0x7ff5700ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff575c11700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5700b2f20 con 0x7ff5700cc540 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff575c11700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5700ce690 con 0x7ff57006ecf0 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff575c11700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57015aa00 con 0x7ff57015a620 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff57015a620 0x7ff5700ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56effd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff5700cc540 0x7ff57006e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57544/0 (socket says 172.21.15.114:57544) 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff57015a620 0x7ff5700ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41834/0 (socket says 172.21.15.114:41834) 2023-11-24T06:27:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56effd700 1 -- 172.21.15.114:0/3413719959 learned_addr learned my addr 172.21.15.114:0/3413719959 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 -- 172.21.15.114:0/3413719959 >> v1:172.21.15.172:6789/0 conn(0x7ff57006ecf0 legacy=0x7ff5700b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 -- 172.21.15.114:0/3413719959 >> v1:172.21.15.114:6789/0 conn(0x7ff5700cc540 legacy=0x7ff57006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 -- 172.21.15.114:0/3413719959 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700ce960 con 0x7ff57015a620 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56e7fc700 1 -- 172.21.15.114:0/3413719959 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636628635 0 0) 0x7ff5700b2f20 con 0x7ff5700cc540 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.614+0000 7ff56f7fe700 1 --2- 172.21.15.114:0/3413719959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff5700ce150 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59488b19ee29f2f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff56e7fc700 1 -- 172.21.15.114:0/3413719959 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011830 con 0x7ff57015a620 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff56e7fc700 1 -- 172.21.15.114:0/3413719959 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558011990 con 0x7ff57015a620 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff56e7fc700 1 -- 172.21.15.114:0/3413719959 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011d20 con 0x7ff57015a620 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 -- 172.21.15.114:0/3413719959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 msgr2=0x7ff5700ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 --2- 172.21.15.114:0/3413719959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff5700ce150 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 -- 172.21.15.114:0/3413719959 shutdown_connections 2023-11-24T06:27:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 --2- 172.21.15.114:0/3413719959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff5700ce150 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 -- 172.21.15.114:0/3413719959 >> 172.21.15.114:0/3413719959 conn(0x7ff5700b2050 msgr2=0x7ff5700b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 -- 172.21.15.114:0/3413719959 shutdown_connections 2023-11-24T06:27:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.615+0000 7ff575c11700 1 -- 172.21.15.114:0/3413719959 wait complete. 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.616+0000 7ff575c11700 1 Processor -- start 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 -- start start 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff57006ecf0 0x7ff570146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5700cc540 0x7ff57013ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff570144e90 con 0x7ff57015a620 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff570145cf0 con 0x7ff57006ecf0 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff575c11700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57013ef40 con 0x7ff5700cc540 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff56ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff56ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41844/0 (socket says 172.21.15.114:41844) 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.617+0000 7ff56ffff700 1 -- 172.21.15.114:0/2656425797 learned_addr learned my addr 172.21.15.114:0/2656425797 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 -- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5700cc540 msgr2=0x7ff57013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5700cc540 0x7ff57013ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 -- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff57006ecf0 msgr2=0x7ff570146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56f7fe700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff57006ecf0 0x7ff570146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff57006ecf0 0x7ff570146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 -- 172.21.15.114:0/2656425797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580094a0 con 0x7ff57015a620 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff56ffff700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff56800cc50 tx=0x7ff56800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56800ade0 con 0x7ff57015a620 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56800ce50 con 0x7ff57015a620 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5701450e0 con 0x7ff57015a620 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568003c20 con 0x7ff57015a620 2023-11-24T06:27:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.618+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5701456a0 con 0x7ff57015a620 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.619+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff570068d90 con 0x7ff57015a620 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.620+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff5680050d0 con 0x7ff57015a620 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.620+0000 7ff567fff700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 0x7ff55006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.620+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff56808a100 con 0x7ff57015a620 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.620+0000 7ff56f7fe700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 0x7ff55006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.620+0000 7ff56f7fe700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 0x7ff55006d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff558009b90 tx=0x7ff55801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.624+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff568055310 con 0x7ff57015a620 2023-11-24T06:27:36.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.813+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff570068fb0 con 0x7ff57015a620 2023-11-24T06:27:36.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.814+0000 7ff567fff700 1 -- 172.21.15.114:0/2656425797 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff56805de60 con 0x7ff57015a620 2023-11-24T06:27:36.817 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 msgr2=0x7ff55006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 0x7ff55006d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff558009b90 tx=0x7ff55801c040 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 msgr2=0x7ff570168420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff56800cc50 tx=0x7ff56800a340 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 shutdown_connections 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff55006b080 0x7ff55006d540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff57015a620 0x7ff570168420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff57006ecf0 0x7ff570146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 --2- 172.21.15.114:0/2656425797 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5700cc540 0x7ff57013ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 >> 172.21.15.114:0/2656425797 conn(0x7ff5700b2050 msgr2=0x7ff5700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 shutdown_connections 2023-11-24T06:27:36.821 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.816+0000 7ff575c11700 1 -- 172.21.15.114:0/2656425797 wait complete. 2023-11-24T06:27:36.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-24T06:27:36.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.949+0000 7f9b2b534700 1 Processor -- start 2023-11-24T06:27:36.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.949+0000 7f9b2b534700 1 -- start start 2023-11-24T06:27:36.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b2b534700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b24158670 0x7f9b240622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b2b534700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b240cfff0 con 0x7f9b24158a50 2023-11-24T06:27:36.957 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b2b534700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b240cfd10 con 0x7f9b240c9fc0 2023-11-24T06:27:36.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b2b534700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b24161f50 con 0x7f9b24158670 2023-11-24T06:27:36.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b28acf700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9b24158a50 0x7f9b240c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57556/0 (socket says 172.21.15.114:57556) 2023-11-24T06:27:36.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b28acf700 1 -- 172.21.15.114:0/941807951 learned_addr learned my addr 172.21.15.114:0/941807951 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:36.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834527419 0 0) 0x7f9b240cfff0 con 0x7f9b24158a50 2023-11-24T06:27:36.958 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b100035b0 con 0x7f9b24158a50 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b292d0700 1 --2- 172.21.15.114:0/941807951 >> v2:172.21.15.172:3300/0 conn(0x7f9b24158670 0x7f9b240622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970251941 0 0) 0x7f9b100035b0 con 0x7f9b24158a50 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 >> v2:172.21.15.172:3300/0 conn(0x7f9b24158670 msgr2=0x7f9b240622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 --2- 172.21.15.114:0/941807951 >> v2:172.21.15.172:3300/0 conn(0x7f9b24158670 0x7f9b240622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 >> v1:172.21.15.172:6789/0 conn(0x7f9b240c9fc0 legacy=0x7f9b240ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b24162230 con 0x7f9b24158a50 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.950+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f9b14001670 con 0x7f9b24158a50 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 >> v1:172.21.15.114:6789/0 conn(0x7f9b24158a50 legacy=0x7f9b240c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b10003a00 0x7f9b10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.959 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 0x7f9b10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10005410 0x7f9b1000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24161f50 con 0x7f9b10004850 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1000c2a0 con 0x7f9b10003a00 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1000c460 con 0x7f9b10005410 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 0x7f9b10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b292d0700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b10003a00 0x7f9b10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b29ad1700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10005410 0x7f9b1000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 -- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10005410 msgr2=0x7f9b1000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10005410 0x7f9b1000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 -- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b10003a00 msgr2=0x7f9b10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:36.960 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b10003a00 0x7f9b10003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 -- 172.21.15.114:0/941807951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100035b0 con 0x7f9b10004850 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b29ad1700 1 -- 172.21.15.114:0/941807951 reap_dead start 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b28acf700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 0x7f9b10004cd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9b2415e870 tx=0x7f9b10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c786dc5b2fc1a075 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14007c10 con 0x7f9b10004850 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1400cd80 con 0x7f9b10004850 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.951+0000 7f9b1bfff700 1 -- 172.21.15.114:0/941807951 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14005ba0 con 0x7f9b10004850 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 -- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 msgr2=0x7f9b10004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 0x7f9b10004cd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9b2415e870 tx=0x7f9b10005880 comp rx=0 tx=0).stop 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 -- 172.21.15.114:0/941807951 shutdown_connections 2023-11-24T06:27:36.961 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 --2- 172.21.15.114:0/941807951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b10004850 0x7f9b10004cd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 -- 172.21.15.114:0/941807951 >> 172.21.15.114:0/941807951 conn(0x7f9b240b57b0 msgr2=0x7f9b240b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 -- 172.21.15.114:0/941807951 shutdown_connections 2023-11-24T06:27:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.952+0000 7f9b2b534700 1 -- 172.21.15.114:0/941807951 wait complete. 2023-11-24T06:27:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.953+0000 7f9b2b534700 1 Processor -- start 2023-11-24T06:27:36.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.953+0000 7f9b2b534700 1 -- start start 2023-11-24T06:27:36.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10004850 0x7f9b241f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b241450d0 0x7f9b241f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240cb580 con 0x7f9b2406dda0 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2406e770 con 0x7f9b241450d0 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24148030 con 0x7f9b10004850 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41862/0 (socket says 172.21.15.114:41862) 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 -- 172.21.15.114:0/4138457046 learned_addr learned my addr 172.21.15.114:0/4138457046 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:36.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b28acf700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b241450d0 0x7f9b241f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b292d0700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10004850 0x7f9b241f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 -- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10004850 msgr2=0x7f9b241f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10004850 0x7f9b241f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 -- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b241450d0 msgr2=0x7f9b241f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b241450d0 0x7f9b241f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 -- 172.21.15.114:0/4138457046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b241f7ed0 con 0x7f9b2406dda0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b29ad1700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9b240b0350 tx=0x7f9b1c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0020e0 con 0x7f9b2406dda0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c003c10 con 0x7f9b2406dda0 2023-11-24T06:27:36.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24147160 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.954+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00d990 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.955+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24147700 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.955+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b240cc940 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.956+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9b1c00daf0 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.956+0000 7f9b19ffb700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 0x7f9b0806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.956+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9b1c089800 con 0x7f9b2406dda0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.956+0000 7f9b292d0700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 0x7f9b0806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.957+0000 7f9b292d0700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 0x7f9b0806d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009950 tx=0x7f9b20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:36.961+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b1c054930 con 0x7f9b2406dda0 2023-11-24T06:27:37.233 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:37.229+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f9b24067f60 con 0x7f9b2406dda0 2023-11-24T06:27:38.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.145+0000 7f9b19ffb700 1 -- 172.21.15.114:0/4138457046 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v187) v1 ==== 164+0+0 (secure 0 0 0) 0x7f9b1c057f50 con 0x7f9b2406dda0 2023-11-24T06:27:38.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 msgr2=0x7f9b0806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 0x7f9b0806d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009950 tx=0x7f9b20008040 comp rx=0 tx=0).stop 2023-11-24T06:27:38.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 msgr2=0x7f9b2406e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9b240b0350 tx=0x7f9b1c00d430 comp rx=0 tx=0).stop 2023-11-24T06:27:38.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 shutdown_connections 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9b0806b080 0x7f9b0806d540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b2406dda0 0x7f9b2406e230 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b10004850 0x7f9b241f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 --2- 172.21.15.114:0/4138457046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b241450d0 0x7f9b241f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.148+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 >> 172.21.15.114:0/4138457046 conn(0x7f9b240b57b0 msgr2=0x7f9b2406ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.149+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 shutdown_connections 2023-11-24T06:27:38.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.149+0000 7f9b2b534700 1 -- 172.21.15.114:0/4138457046 wait complete. 2023-11-24T06:27:38.167 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:27:38.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.292+0000 7f3f32735700 1 Processor -- start 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.292+0000 7f3f32735700 1 -- start start 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f32735700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f2c0d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f32735700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f2c071d20 0x7f3f2c0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f32735700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2c0d2a60 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f32735700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2c0d2d40 con 0x7f3f2c071d20 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f32735700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0d3020 con 0x7f3f2c15e090 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2b7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3f2c0d24e0 0x7f3f2c071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57576/0 (socket says 172.21.15.114:57576) 2023-11-24T06:27:38.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2b7fe700 1 -- 172.21.15.114:0/1245033074 learned_addr learned my addr 172.21.15.114:0/1245033074 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301662971 0 0) 0x7f3f2c0d2a60 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f140035b0 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f30cd2700 1 --2- 172.21.15.114:0/1245033074 >> v2:172.21.15.172:3300/0 conn(0x7f3f2c071d20 0x7f3f2c0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2bfff700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f2c0d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816969931 0 0) 0x7f3f140035b0 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 msgr2=0x7f3f2c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f2c0d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 >> v2:172.21.15.172:3300/0 conn(0x7f3f2c071d20 msgr2=0x7f3f2c0b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 --2- 172.21.15.114:0/1245033074 >> v2:172.21.15.172:3300/0 conn(0x7f3f2c071d20 0x7f3f2c0b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0d3300 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.293+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f3f1c001670 con 0x7f3f2c0d24e0 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 >> v1:172.21.15.114:6789/0 conn(0x7f3f2c0d24e0 legacy=0x7f3f2c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f14003f80 0x7f3f14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f14004d90 0x7f3f14005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0d2d40 con 0x7f3f14003f80 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0d3020 con 0x7f3f14004d90 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1400ba40 con 0x7f3f2c15e090 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f14003f80 0x7f3f14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2bfff700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f30cd2700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f14004d90 0x7f3f14005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 -- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 msgr2=0x7f3f14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f2c15e090 0x7f3f14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 -- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f14004d90 msgr2=0x7f3f14005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f14004d90 0x7f3f14005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 -- 172.21.15.114:0/1245033074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140035b0 con 0x7f3f14003f80 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2b7fe700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f14003f80 0x7f3f14004850 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c320 tx=0x7f3f14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed95396e09b3f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c00ce60 con 0x7f3f14003f80 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c005960 con 0x7f3f14003f80 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.294+0000 7f3f2affd700 1 -- 172.21.15.114:0/1245033074 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c005cf0 con 0x7f3f14003f80 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 -- 172.21.15.114:0/1245033074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f14003f80 msgr2=0x7f3f14004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 --2- 172.21.15.114:0/1245033074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f14003f80 0x7f3f14004850 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00c320 tx=0x7f3f14005880 comp rx=0 tx=0).stop 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f30cd2700 1 -- 172.21.15.114:0/1245033074 reap_dead start 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 -- 172.21.15.114:0/1245033074 shutdown_connections 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 -- 172.21.15.114:0/1245033074 >> 172.21.15.114:0/1245033074 conn(0x7f3f2c06b410 msgr2=0x7f3f2c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 -- 172.21.15.114:0/1245033074 shutdown_connections 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.295+0000 7f3f32735700 1 -- 172.21.15.114:0/1245033074 wait complete. 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.296+0000 7f3f32735700 1 Processor -- start 2023-11-24T06:27:38.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.296+0000 7f3f32735700 1 -- start start 2023-11-24T06:27:38.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f14003f80 0x7f3f2c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f2c0cb8c0 0x7f3f2c0ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c160430 con 0x7f3f2c15fa60 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c15a850 con 0x7f3f2c0cb8c0 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f32735700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c0aa810 con 0x7f3f14003f80 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41882/0 (socket says 172.21.15.114:41882) 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 -- 172.21.15.114:0/2674081080 learned_addr learned my addr 172.21.15.114:0/2674081080 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f2bfff700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f14003f80 0x7f3f2c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f2b7fe700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f2c0cb8c0 0x7f3f2c0ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 -- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f14003f80 msgr2=0x7f3f2c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f14003f80 0x7f3f2c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 -- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f2c0cb8c0 msgr2=0x7f3f2c0ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.313 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f2c0cb8c0 0x7f3f2c0ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.297+0000 7f3f30cd2700 1 -- 172.21.15.114:0/2674081080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c0d3c80 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f30cd2700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0b6f00 tx=0x7f3f24011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400e040 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400f040 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400fc70 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c0ab7c0 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.298+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c0aad40 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.299+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2c0aaf60 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.299+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3f24015430 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.299+0000 7f3f28ff9700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 0x7f3f1806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.300+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f24089d90 con 0x7f3f2c15fa60 2023-11-24T06:27:38.314 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.300+0000 7f3f2bfff700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 0x7f3f1806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.300+0000 7f3f2bfff700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 0x7f3f1806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c15ab00 tx=0x7f3f20009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.304+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f24054e40 con 0x7f3f2c15fa60 2023-11-24T06:27:38.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.495+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3f2c068fb0 con 0x7f3f2c15fa60 2023-11-24T06:27:38.502 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.498+0000 7f3f28ff9700 1 -- 172.21.15.114:0/2674081080 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f24058460 con 0x7f3f2c15fa60 2023-11-24T06:27:38.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 msgr2=0x7f3f1806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.504 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 0x7f3f1806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c15ab00 tx=0x7f3f20009210 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 msgr2=0x7f3f2c15fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0b6f00 tx=0x7f3f24011ed0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 shutdown_connections 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3f1806b080 0x7f3f1806d540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3f2c15fa60 0x7f3f2c15fef0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f14003f80 0x7f3f2c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 --2- 172.21.15.114:0/2674081080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f2c0cb8c0 0x7f3f2c0ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 >> 172.21.15.114:0/2674081080 conn(0x7f3f2c06b410 msgr2=0x7f3f2c0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 shutdown_connections 2023-11-24T06:27:38.505 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.501+0000 7f3f32735700 1 -- 172.21.15.114:0/2674081080 wait complete. 2023-11-24T06:27:38.516 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:27:38.517 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:38.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:38.612 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:27:38.612 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:38.612 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:38.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:27:38.630 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:27:38.631 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:38.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:27:38.672 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:38.672 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:38.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:27:38.689 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:27:38.689 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:38.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:27:38.732 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:27:38.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:38.853 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.850+0000 7f3bb024b700 1 Processor -- start 2023-11-24T06:27:38.853 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.850+0000 7f3bb024b700 1 -- start start 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bb024b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ba815bdd0 0x7f3ba81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bb024b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba80b3370 0x7f3ba80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bb024b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80b2030 con 0x7f3ba80d1c70 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bb024b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80b2f50 con 0x7f3ba815bdd0 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ba815bdd0 0x7f3ba81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bb024b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80b22d0 con 0x7f3ba80b3370 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ba815bdd0 0x7f3ba81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41886/0 (socket says 172.21.15.114:41886) 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 -- 172.21.15.114:0/1339890513 learned_addr learned my addr 172.21.15.114:0/1339890513 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:38.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 -- 172.21.15.114:0/1339890513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba80b3370 msgr2=0x7f3ba80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 --2- 172.21.15.114:0/1339890513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba80b3370 0x7f3ba80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 -- 172.21.15.114:0/1339890513 >> v1:172.21.15.114:6789/0 conn(0x7f3ba80d1c70 legacy=0x7f3ba80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 -- 172.21.15.114:0/1339890513 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80b25b0 con 0x7f3ba815bdd0 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3badfe7700 1 --2- 172.21.15.114:0/1339890513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba815bdd0 0x7f3ba81604e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40098f0 tx=0x7f3ba4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4c7886bbcb5c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bacfe5700 1 -- 172.21.15.114:0/1339890513 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400b8a0 con 0x7f3ba815bdd0 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bacfe5700 1 -- 172.21.15.114:0/1339890513 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba400ba00 con 0x7f3ba815bdd0 2023-11-24T06:27:38.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.851+0000 7f3bacfe5700 1 -- 172.21.15.114:0/1339890513 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400bd90 con 0x7f3ba815bdd0 2023-11-24T06:27:38.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 -- 172.21.15.114:0/1339890513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba815bdd0 msgr2=0x7f3ba81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 --2- 172.21.15.114:0/1339890513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba815bdd0 0x7f3ba81604e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40098f0 tx=0x7f3ba4009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 -- 172.21.15.114:0/1339890513 shutdown_connections 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 --2- 172.21.15.114:0/1339890513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba815bdd0 0x7f3ba81604e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 --2- 172.21.15.114:0/1339890513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba80b3370 0x7f3ba80b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 -- 172.21.15.114:0/1339890513 >> 172.21.15.114:0/1339890513 conn(0x7f3ba80731a0 msgr2=0x7f3ba809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 -- 172.21.15.114:0/1339890513 shutdown_connections 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.852+0000 7f3bb024b700 1 -- 172.21.15.114:0/1339890513 wait complete. 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.853+0000 7f3bb024b700 1 Processor -- start 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.853+0000 7f3bb024b700 1 -- start start 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba80b3370 0x7f3ba815ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba815bdd0 0x7f3ba80cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba815f4d0 con 0x7f3ba80d1c70 2023-11-24T06:27:38.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80caf40 con 0x7f3ba80b3370 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bb024b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ca330 con 0x7f3ba815bdd0 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41894/0 (socket says 172.21.15.114:41894) 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 -- 172.21.15.114:0/4254133877 learned_addr learned my addr 172.21.15.114:0/4254133877 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 -- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba815bdd0 msgr2=0x7f3ba80cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba815bdd0 0x7f3ba80cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 -- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba80b3370 msgr2=0x7f3ba815ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba80b3370 0x7f3ba815ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.854+0000 7f3bad7e6700 1 -- 172.21.15.114:0/4254133877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009550 con 0x7f3ba80d1c70 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3bad7e6700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b530 tx=0x7f3b9800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98011830 con 0x7f3ba80d1c70 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b980119b0 con 0x7f3ba80d1c70 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80ca610 con 0x7f3ba80d1c70 2023-11-24T06:27:38.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98011d60 con 0x7f3ba80d1c70 2023-11-24T06:27:38.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.855+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80cabd0 con 0x7f3ba80d1c70 2023-11-24T06:27:38.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.856+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8070480 con 0x7f3ba80d1c70 2023-11-24T06:27:38.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.856+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3b9801a400 con 0x7f3ba80d1c70 2023-11-24T06:27:38.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.856+0000 7f3b9e7fc700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 0x7f3b8c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.856+0000 7f3badfe7700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 0x7f3b8c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.857+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3b980895e0 con 0x7f3ba80d1c70 2023-11-24T06:27:38.862 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.857+0000 7f3badfe7700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 0x7f3b8c06d540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4014040 tx=0x7f3ba4010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:38.861+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b980547a0 con 0x7f3ba80d1c70 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.095+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba80cb080 con 0x7f3ba80d1c70 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.095+0000 7f3b9e7fc700 1 -- 172.21.15.114:0/4254133877 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3ba80cb080 con 0x7f3ba80d1c70 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1770373197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1770373197},{"type":"v1","addr":"172.21.15.172:6834","nonce":1770373197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3042215503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3042215503},{"type":"v1","addr":"172.21.15.172:6835","nonce":3042215503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/372036441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":372036441},{"type":"v1","addr":"172.21.15.114:6836","nonce":372036441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6141,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/327220376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":327220376},{"type":"v1","addr":"172.21.15.114:6837","nonce":327220376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-11-24T06:27:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 msgr2=0x7f3b8c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 0x7f3b8c06d540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4014040 tx=0x7f3ba4010000 comp rx=0 tx=0).stop 2023-11-24T06:27:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 msgr2=0x7f3ba81566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b530 tx=0x7f3b9800b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 shutdown_connections 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3b8c06b080 0x7f3b8c06d540 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ba815bdd0 0x7f3ba80cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba80b3370 0x7f3ba815ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 --2- 172.21.15.114:0/4254133877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba80d1c70 0x7f3ba81566a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 >> 172.21.15.114:0/4254133877 conn(0x7f3ba80731a0 msgr2=0x7f3ba806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 shutdown_connections 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.097+0000 7f3bb024b700 1 -- 172.21.15.114:0/4254133877 wait complete. 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2023-11-24T06:27:39.114 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:27:39.114 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:27:39.114 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:27:39.114 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:39.115 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:27:39.115 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:27:39.115 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:27:39.115 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:39.115 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:27:39.115 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:27:39.116 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:27:39.116 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:39.116 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:27:39.116 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:27:39.116 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:27:39.116 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:39.126 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:27:39.126 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:27:39.127 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:27:39.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:27:39.128 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:27:39.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:27:39.129 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:27:39.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:27:39.169 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:27:39.169 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:39.170 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:27:39.171 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:39.171 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:27:39.172 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:39.172 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:27:39.172 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:39.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:27:39.247 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2023-11-24T06:27:39.248 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:27:39.257 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2023-11-24T06:27:39.257 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2023-11-24T06:27:39.257 INFO:tasks.ceph.mds.b.smithi172.stderr:2023-11-24T06:27:39.255+0000 7f47d89dc700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-24T06:27:39.299 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.296+0000 7fefb4733700 1 Processor -- start 2023-11-24T06:27:39.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.296+0000 7fefb4733700 1 -- start start 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb4733700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fefac13e000 0x7fefac15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb4733700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0b4290 con 0x7fefac0cff10 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb4733700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac13e570 con 0x7fefac15efb0 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb4733700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefac15f430 con 0x7fefac13e000 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb24cf700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fefac13e000 0x7fefac15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb2cd0700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fefac0cff10 0x7fefac0b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57634/0 (socket says 172.21.15.114:57634) 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb2cd0700 1 -- 172.21.15.114:0/3114855768 learned_addr learned my addr 172.21.15.114:0/3114855768 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:39.301 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb24cf700 1 -- 172.21.15.114:0/3114855768 >> v1:172.21.15.172:6789/0 conn(0x7fefac15efb0 legacy=0x7fefac0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb24cf700 1 -- 172.21.15.114:0/3114855768 >> v1:172.21.15.114:6789/0 conn(0x7fefac0cff10 legacy=0x7fefac0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.297+0000 7fefb24cf700 1 -- 172.21.15.114:0/3114855768 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0d0330 con 0x7fefac13e000 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb24cf700 1 --2- 172.21.15.114:0/3114855768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac13e000 0x7fefac15ea70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009840 tx=0x7fefa8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65da9ea4d061b9c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb14cd700 1 -- 172.21.15.114:0/3114855768 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8011830 con 0x7fefac13e000 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb14cd700 1 -- 172.21.15.114:0/3114855768 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa8011990 con 0x7fefac13e000 2023-11-24T06:27:39.302 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb14cd700 1 -- 172.21.15.114:0/3114855768 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8011d20 con 0x7fefac13e000 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 -- 172.21.15.114:0/3114855768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac13e000 msgr2=0x7fefac15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 --2- 172.21.15.114:0/3114855768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac13e000 0x7fefac15ea70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009840 tx=0x7fefa8009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 -- 172.21.15.114:0/3114855768 shutdown_connections 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 --2- 172.21.15.114:0/3114855768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac13e000 0x7fefac15ea70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 -- 172.21.15.114:0/3114855768 >> 172.21.15.114:0/3114855768 conn(0x7fefac072990 msgr2=0x7fefac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 -- 172.21.15.114:0/3114855768 shutdown_connections 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.298+0000 7fefb4733700 1 -- 172.21.15.114:0/3114855768 wait complete. 2023-11-24T06:27:39.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.299+0000 7fefb4733700 1 Processor -- start 2023-11-24T06:27:39.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.299+0000 7fefb4733700 1 -- start start 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefac0cff10 0x7fefac139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefac13e000 0x7fefac1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac143dd0 con 0x7fefac15efb0 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac13fa90 con 0x7fefac13e000 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb4733700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac143150 con 0x7fefac0cff10 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb2cd0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb2cd0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41932/0 (socket says 172.21.15.114:41932) 2023-11-24T06:27:39.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb2cd0700 1 -- 172.21.15.114:0/1941071185 learned_addr learned my addr 172.21.15.114:0/1941071185 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb24cf700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefac0cff10 0x7fefac139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.300+0000 7fefb1cce700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefac13e000 0x7fefac1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 -- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefac0cff10 msgr2=0x7fefac139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefac0cff10 0x7fefac139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 -- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefac13e000 msgr2=0x7fefac1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefac13e000 0x7fefac1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 -- 172.21.15.114:0/1941071185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80094a0 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb2cd0700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fefa400cbf0 tx=0x7fefa400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400aa60 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa400abc0 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4012400 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac143430 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.301+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac1439f0 con 0x7fefac15efb0 2023-11-24T06:27:39.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.302+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac143f10 con 0x7fefac15efb0 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.302+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fefa401b3b0 con 0x7fefac15efb0 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.303+0000 7fefa2ffd700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 0x7fef9006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.303+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fefa40890e0 con 0x7fefac15efb0 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.303+0000 7fefb24cf700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 0x7fef9006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.303+0000 7fefb24cf700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 0x7fef9006d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fefa80090b0 tx=0x7fefa8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.307+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefac143f10 con 0x7fefac15efb0 2023-11-24T06:27:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:39.512+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fefac143f10 con 0x7fefac15efb0 2023-11-24T06:27:40.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.157+0000 7fefa2ffd700 1 -- 172.21.15.114:0/1941071185 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7fefac143f10 con 0x7fefac15efb0 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 msgr2=0x7fef9006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 0x7fef9006d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fefa80090b0 tx=0x7fefa8008030 comp rx=0 tx=0).stop 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 msgr2=0x7fefac152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fefa400cbf0 tx=0x7fefa400cf00 comp rx=0 tx=0).stop 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 shutdown_connections 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fef9006b080 0x7fef9006d540 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefac15efb0 0x7fefac152ae0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefac0cff10 0x7fefac139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 --2- 172.21.15.114:0/1941071185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefac13e000 0x7fefac1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 >> 172.21.15.114:0/1941071185 conn(0x7fefac072990 msgr2=0x7fefac0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 shutdown_connections 2023-11-24T06:27:40.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.160+0000 7fefb4733700 1 -- 172.21.15.114:0/1941071185 wait complete. 2023-11-24T06:27:40.166 INFO:teuthology.orchestra.run.smithi114.stderr: removed all blocklist entries 2023-11-24T06:27:40.178 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:27:40.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.296+0000 7f69ab72e700 1 Processor -- start 2023-11-24T06:27:40.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.297+0000 7f69ab72e700 1 -- start start 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.297+0000 7f69ab72e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f69a40adf00 0x7f69a406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.297+0000 7f69ab72e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a40af950 con 0x7f69a41446a0 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.297+0000 7f69ab72e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a40afc30 con 0x7f69a406c540 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.297+0000 7f69ab72e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a406c9f0 con 0x7f69a40adf00 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a94ca700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f69a41446a0 0x7f69a40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57646/0 (socket says 172.21.15.114:57646) 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a94ca700 1 -- 172.21.15.114:0/2379518227 learned_addr learned my addr 172.21.15.114:0/2379518227 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423171595 0 0) 0x7f69a40af950 con 0x7f69a41446a0 2023-11-24T06:27:40.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69900035b0 con 0x7f69a41446a0 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a8cc9700 1 --2- 172.21.15.114:0/2379518227 >> v2:172.21.15.172:3300/0 conn(0x7f69a40adf00 0x7f69a406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447202463 0 0) 0x7f69900035b0 con 0x7f69a41446a0 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 >> v2:172.21.15.172:3300/0 conn(0x7f69a40adf00 msgr2=0x7f69a406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 --2- 172.21.15.114:0/2379518227 >> v2:172.21.15.172:3300/0 conn(0x7f69a40adf00 0x7f69a406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 >> v1:172.21.15.172:6789/0 conn(0x7f69a406c540 legacy=0x7f69a40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a40aff10 con 0x7f69a41446a0 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f6994001670 con 0x7f69a41446a0 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 >> v1:172.21.15.114:6789/0 conn(0x7f69a41446a0 legacy=0x7f69a40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6990003a00 0x7f6990003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6990004850 0x7f6990004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f699000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a406c9f0 con 0x7f6990005210 2023-11-24T06:27:40.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699000c0b0 con 0x7f6990003a00 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699000c270 con 0x7f6990004850 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a9ccb700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f699000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a8cc9700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6990004850 0x7f6990004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.298+0000 7f69a94ca700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6990003a00 0x7f6990003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 -- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6990004850 msgr2=0x7f6990004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6990004850 0x7f6990004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 -- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6990003a00 msgr2=0x7f6990003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6990003a00 0x7f6990003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 -- 172.21.15.114:0/2379518227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900035b0 con 0x7f6990005210 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 -- 172.21.15.114:0/2379518227 reap_dead start 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69a9ccb700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f699000ba10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f699c00c190 tx=0x7f6990005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d01d249512c83d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.306 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c001db0 con 0x7f6990005210 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c00ccf0 con 0x7f6990005210 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f699bfff700 1 -- 172.21.15.114:0/2379518227 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c003c40 con 0x7f6990005210 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69ab72e700 1 -- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 msgr2=0x7f699000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f699000ba10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f699c00c190 tx=0x7f6990005880 comp rx=0 tx=0).stop 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69ab72e700 1 -- 172.21.15.114:0/2379518227 shutdown_connections 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2379518227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f699000ba10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.299+0000 7f69ab72e700 1 -- 172.21.15.114:0/2379518227 >> 172.21.15.114:0/2379518227 conn(0x7f69a4071b90 msgr2=0x7f69a4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.300+0000 7f69ab72e700 1 -- 172.21.15.114:0/2379518227 shutdown_connections 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.300+0000 7f69ab72e700 1 -- 172.21.15.114:0/2379518227 wait complete. 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 Processor -- start 2023-11-24T06:27:40.307 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 -- start start 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a4140870 0x7f69a4140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69a41f7220 0x7f69a41f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4141210 con 0x7f6990005210 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a41f7f70 con 0x7f69a4140870 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.301+0000 7f69ab72e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a41f3f10 con 0x7f69a41f7220 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41950/0 (socket says 172.21.15.114:41950) 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 -- 172.21.15.114:0/2153528180 learned_addr learned my addr 172.21.15.114:0/2153528180 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a9ccb700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69a41f7220 0x7f69a41f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 -- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69a41f7220 msgr2=0x7f69a41f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a8cc9700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a4140870 0x7f69a4140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69a41f7220 0x7f69a41f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 -- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a4140870 msgr2=0x7f69a4140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a4140870 0x7f69a4140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 -- 172.21.15.114:0/2153528180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a41f41f0 con 0x7f6990005210 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69a94ca700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6994007290 tx=0x7f69940075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699400f660 con 0x7f6990005210 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699400f7c0 con 0x7f6990005210 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699400fb70 con 0x7f6990005210 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a41f44d0 con 0x7f6990005210 2023-11-24T06:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.302+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a41f7bf0 con 0x7f6990005210 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.303+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a40753f0 con 0x7f6990005210 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.304+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f699400fcd0 con 0x7f6990005210 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.304+0000 7f6999ffb700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 0x7f698806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.304+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69940895a0 con 0x7f6990005210 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.304+0000 7f69a8cc9700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 0x7f698806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.304+0000 7f69a8cc9700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 0x7f698806d540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69a406f1d0 tx=0x7f69a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.312 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.308+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6994054720 con 0x7f6990005210 2023-11-24T06:27:40.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.578+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f69a4067f60 con 0x7f6990005210 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.578+0000 7f6999ffb700 1 -- 172.21.15.114:0/2153528180 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6994057d40 con 0x7f6990005210 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout: "auth_dump": [ 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.a", 2023-11-24T06:27:40.582 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBloXHaIBAA40BWWVcYcVbX+kM3wd4srw==", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.b", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlRJM0LRAAToQLwjepMjKSY6cakPeFJw==", 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.583 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.c", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBllY8dKBAAtxCS400KLv4rt6WIzc3a2g==", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mds.d", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlwqqDNBAARSrqgGNEALaFrG0ZFStYgg==", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.0", 2023-11-24T06:27:40.585 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBZQWBlQ7/gFxAA603E5V+l/7dkugKMUNQV3A==", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.1", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBcQWBlnHtEFxAAr9h0RnKScdwcnLc2O9Pp4A==", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.586 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.2", 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBfQWBlmYVaGBAARyCei1YbVxxzy00k0W/Z6w==", 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.587 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.3", 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBjQWBlVqdEABAAyD3s0xW8jXecRwdgtq8AwQ==", 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.4", 2023-11-24T06:27:40.588 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBpQWBlyLk4BxAAauMGnn0akTG5zKYKcy9Vmw==", 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.5", 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBsQWBlQjG4BhAAuv1Ja24nPcZ+SmnV8yuHgg==", 2023-11-24T06:27:40.589 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.6", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBvQWBl4tb6BRAAcoydr7CybV8Ckv++5LFS+w==", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "osd.7", 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQByQWBlIUmuBRAAO0eSyPnvmpUft7iB4vQ6dg==", 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow profile osd", 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile osd", 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.591 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.0", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g==", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow r", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.1", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlSFhWBhAA13v9kza/+snKVCmwnfK4eA==", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.592 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.2", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlKtb6AhAAoaQhrX2DZgvNW0rcCB1iVg==", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:27:40.593 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.3", 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBWQWBlwshBDBAAkMjk+usVBY7TYehxOhJdDw==", 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow", 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow r", 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow rw", 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow rwx" 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "client.admin", 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlz1e2AhAAHaN4NjkwUY+J8W2nidj3Gg==", 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr": "allow *", 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow *", 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.595 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.x", 2023-11-24T06:27:40.596 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlT3B0HRAANmHJk8Er7FHsAeQJuAnUKg==", 2023-11-24T06:27:40.596 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.596 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:27:40.596 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:27:40.596 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: "entity": "mgr.y", 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "AQBVQWBlcY5dGBAAZ7ISSGzT/OCSkRVGTBwkvA==", 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": { 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": "allow *", 2023-11-24T06:27:40.597 INFO:teuthology.orchestra.run.smithi114.stdout: "mon": "allow profile mgr", 2023-11-24T06:27:40.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "allow *" 2023-11-24T06:27:40.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:40.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2023-11-24T06:27:40.598 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 msgr2=0x7f698806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 0x7f698806d540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f69a406f1d0 tx=0x7f69a0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 msgr2=0x7f69a40b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6994007290 tx=0x7f69940075a0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 shutdown_connections 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f698806b080 0x7f698806d540 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69a41f7220 0x7f69a41f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.580+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6990005210 0x7f69a40b1b30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.581+0000 7f69ab72e700 1 --2- 172.21.15.114:0/2153528180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a4140870 0x7f69a4140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.581+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 >> 172.21.15.114:0/2153528180 conn(0x7f69a4071b90 msgr2=0x7f69a40b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.581+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 shutdown_connections 2023-11-24T06:27:40.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.581+0000 7f69ab72e700 1 -- 172.21.15.114:0/2153528180 wait complete. 2023-11-24T06:27:40.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:40.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.718+0000 7fdcb03d9700 1 Processor -- start 2023-11-24T06:27:40.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.718+0000 7fdcb03d9700 1 -- start start 2023-11-24T06:27:40.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.719+0000 7fdcb03d9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdca80ce140 0x7fdca815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.719+0000 7fdcb03d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.719+0000 7fdcb03d9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca81414c0 con 0x7fdca815af10 2023-11-24T06:27:40.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.719+0000 7fdcb03d9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca815c300 con 0x7fdca80ce140 2023-11-24T06:27:40.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.719+0000 7fdcb03d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80ce9f0 con 0x7fdca815f630 2023-11-24T06:27:40.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcad974700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdca80ce140 0x7fdca815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36506/0 (socket says 172.21.15.114:36506) 2023-11-24T06:27:40.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 -- 172.21.15.114:0/1128808835 learned_addr learned my addr 172.21.15.114:0/1128808835 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 -- 172.21.15.114:0/1128808835 >> v2:172.21.15.172:3300/0 conn(0x7fdca80ce140 msgr2=0x7fdca815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 --2- 172.21.15.114:0/1128808835 >> v2:172.21.15.172:3300/0 conn(0x7fdca80ce140 0x7fdca815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 -- 172.21.15.114:0/1128808835 >> v1:172.21.15.172:6789/0 conn(0x7fdca815af10 legacy=0x7fdca815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 -- 172.21.15.114:0/1128808835 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80cecd0 con 0x7fdca815f630 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcae976700 1 --2- 172.21.15.114:0/1128808835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009e50 tx=0x7fdca000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3251664c330ed25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcad173700 1 -- 172.21.15.114:0/1128808835 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000e070 con 0x7fdca815f630 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcad173700 1 -- 172.21.15.114:0/1128808835 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000b910 con 0x7fdca815f630 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.720+0000 7fdcad173700 1 -- 172.21.15.114:0/1128808835 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000bcc0 con 0x7fdca815f630 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1128808835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 msgr2=0x7fdca806b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1128808835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009e50 tx=0x7fdca000b330 comp rx=0 tx=0).stop 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1128808835 shutdown_connections 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1128808835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815f630 0x7fdca806b640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1128808835 >> v2:172.21.15.172:3300/0 conn(0x7fdca80ce140 0x7fdca815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1128808835 >> 172.21.15.114:0/1128808835 conn(0x7fdca80cd570 msgr2=0x7fdca80cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1128808835 shutdown_connections 2023-11-24T06:27:40.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.721+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1128808835 wait complete. 2023-11-24T06:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.722+0000 7fdcb03d9700 1 Processor -- start 2023-11-24T06:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.722+0000 7fdcb03d9700 1 -- start start 2023-11-24T06:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca80ce140 0x7fdca8141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815af10 0x7fdca813d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80a9460 con 0x7fdca81f0dd0 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b03a0 con 0x7fdca80ce140 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcb03d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8141e70 con 0x7fdca815af10 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41968/0 (socket says 172.21.15.114:41968) 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 -- 172.21.15.114:0/1479076122 learned_addr learned my addr 172.21.15.114:0/1479076122 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae175700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca80ce140 0x7fdca8141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcad974700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815af10 0x7fdca813d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 -- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815af10 msgr2=0x7fdca813d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815af10 0x7fdca813d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 -- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca80ce140 msgr2=0x7fdca8141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca80ce140 0x7fdca8141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.723+0000 7fdcae976700 1 -- 172.21.15.114:0/1479076122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009b30 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdcae976700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdca815c440 tx=0x7fdca0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0004700 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0004860 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca8142150 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca001c780 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.724+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca8142710 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.725+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8067f60 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.725+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fdca0022070 con 0x7fdca81f0dd0 2023-11-24T06:27:40.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.725+0000 7fdc9effd700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 0x7fdc8c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.726+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdca008d070 con 0x7fdca81f0dd0 2023-11-24T06:27:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.726+0000 7fdcae175700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 0x7fdc8c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.726+0000 7fdcae175700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 0x7fdc8c06d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc980016c0 tx=0x7fdc98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.730+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdca0058280 con 0x7fdca81f0dd0 2023-11-24T06:27:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.919+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdca8072e00 con 0x7fdca81f0dd0 2023-11-24T06:27:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.920+0000 7fdc9effd700 1 -- 172.21.15.114:0/1479076122 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12893 (secure 0 0 0) 0x7fdca005b8a0 con 0x7fdca81f0dd0 2023-11-24T06:27:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:40.924 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":54,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:40.165235+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 msgr2=0x7fdc8c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 0x7fdc8c06d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc980016c0 tx=0x7fdc98009040 comp rx=0 tx=0).stop 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 msgr2=0x7fdca81f1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdca815c440 tx=0x7fdca0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 shutdown_connections 2023-11-24T06:27:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdc8c06b080 0x7fdc8c06d540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdca81f0dd0 0x7fdca81f1280 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca80ce140 0x7fdca8141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 --2- 172.21.15.114:0/1479076122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdca815af10 0x7fdca813d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 >> 172.21.15.114:0/1479076122 conn(0x7fdca80cd570 msgr2=0x7fdca813fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 shutdown_connections 2023-11-24T06:27:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:40.922+0000 7fdcb03d9700 1 -- 172.21.15.114:0/1479076122 wait complete. 2023-11-24T06:27:40.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:40.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:41.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 Processor -- start 2023-11-24T06:27:41.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 -- start start 2023-11-24T06:27:41.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00746a0 0x7f38b00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38b00afc90 0x7f38b006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00b3c60 con 0x7f38b015e160 2023-11-24T06:27:41.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b00b4500 con 0x7f38b00afc90 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.060+0000 7f38b6fa9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00b58c0 con 0x7f38b00746a0 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38b00afc90 0x7f38b006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38b00afc90 0x7f38b006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41976/0 (socket says 172.21.15.114:41976) 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 -- 172.21.15.114:0/1176810882 learned_addr learned my addr 172.21.15.114:0/1176810882 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38affff700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00746a0 0x7f38b00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 -- 172.21.15.114:0/1176810882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00746a0 msgr2=0x7f38b00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00746a0 0x7f38b00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 -- 172.21.15.114:0/1176810882 >> v1:172.21.15.172:6789/0 conn(0x7f38b015e160 legacy=0x7f38b0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 -- 172.21.15.114:0/1176810882 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00b0140 con 0x7f38b00afc90 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38b5546700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00afc90 0x7f38b006d930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e50 tx=0x7f38a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8fe042815c9f83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38af7fe700 1 -- 172.21.15.114:0/1176810882 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800e070 con 0x7f38b00afc90 2023-11-24T06:27:41.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38af7fe700 1 -- 172.21.15.114:0/1176810882 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a800b910 con 0x7f38b00afc90 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.061+0000 7f38af7fe700 1 -- 172.21.15.114:0/1176810882 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800bcc0 con 0x7f38b00afc90 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 -- 172.21.15.114:0/1176810882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00afc90 msgr2=0x7f38b006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00afc90 0x7f38b006d930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e50 tx=0x7f38a800b330 comp rx=0 tx=0).stop 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 -- 172.21.15.114:0/1176810882 shutdown_connections 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00afc90 0x7f38b006d930 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/1176810882 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00746a0 0x7f38b00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 -- 172.21.15.114:0/1176810882 >> 172.21.15.114:0/1176810882 conn(0x7f38b00d0bf0 msgr2=0x7f38b00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 -- 172.21.15.114:0/1176810882 shutdown_connections 2023-11-24T06:27:41.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.062+0000 7f38b6fa9700 1 -- 172.21.15.114:0/1176810882 wait complete. 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.063+0000 7f38b6fa9700 1 Processor -- start 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.063+0000 7f38b6fa9700 1 -- start start 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00afc90 0x7f38b01f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b015e160 0x7f38b00add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01f4c60 con 0x7f38b00746a0 2023-11-24T06:27:41.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01f7c00 con 0x7f38b015e160 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b6fa9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01f7ee0 con 0x7f38b00afc90 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41978/0 (socket says 172.21.15.114:41978) 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 -- 172.21.15.114:0/691814863 learned_addr learned my addr 172.21.15.114:0/691814863 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 -- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00afc90 msgr2=0x7f38b01f7140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00afc90 0x7f38b01f7140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 -- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b015e160 msgr2=0x7f38b00add40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:27:41.071 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b015e160 0x7f38b00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 -- 172.21.15.114:0/691814863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009b30 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38b4d45700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40052d0 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.064+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4005430 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.065+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01f4f20 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.065+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40038c0 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.065+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b015af90 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.066+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0068fb0 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.066+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f38a4003a20 con 0x7f38b00746a0 2023-11-24T06:27:41.073 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.066+0000 7f38ad7fa700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 0x7f389406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.066+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38a4088870 con 0x7f38b00746a0 2023-11-24T06:27:41.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.066+0000 7f38affff700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 0x7f389406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.074 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.067+0000 7f38affff700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 0x7f389406d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38b006f380 tx=0x7f389c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.071+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38a4053a80 con 0x7f38b00746a0 2023-11-24T06:27:41.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.261+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38b006c140 con 0x7f38b00746a0 2023-11-24T06:27:41.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.261+0000 7f38ad7fa700 1 -- 172.21.15.114:0/691814863 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38a40570a0 con 0x7f38b00746a0 2023-11-24T06:27:41.265 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 msgr2=0x7f389406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 0x7f389406d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38b006f380 tx=0x7f389c009210 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 msgr2=0x7f38b015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c810 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 shutdown_connections 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f389406b080 0x7f389406d540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b015e160 0x7f38b00add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38b00746a0 0x7f38b015aa50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 --2- 172.21.15.114:0/691814863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38b00afc90 0x7f38b01f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 >> 172.21.15.114:0/691814863 conn(0x7f38b00d0bf0 msgr2=0x7f38b00725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 shutdown_connections 2023-11-24T06:27:41.268 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.263+0000 7f38b6fa9700 1 -- 172.21.15.114:0/691814863 wait complete. 2023-11-24T06:27:41.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:27:41.398 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.395+0000 7fa94212b700 1 Processor -- start 2023-11-24T06:27:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.395+0000 7fa94212b700 1 -- start start 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa94212b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c15e080 0x7fa93c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa94212b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa93c06c710 0x7fa93c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa94212b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c0ae430 con 0x7fa93c0af4d0 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa94212b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c0af0b0 con 0x7fa93c06c710 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa94212b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0b05a0 con 0x7fa93c15e080 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa93c06c710 0x7fa93c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93affd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa93c0af4d0 0x7fa93c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57698/0 (socket says 172.21.15.114:57698) 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93affd700 1 -- 172.21.15.114:0/1597950124 learned_addr learned my addr 172.21.15.114:0/1597950124 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93b7fe700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c15e080 0x7fa93c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93bfff700 1 -- 172.21.15.114:0/1597950124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c15e080 msgr2=0x7fa93c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93bfff700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c15e080 0x7fa93c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93bfff700 1 -- 172.21.15.114:0/1597950124 >> v1:172.21.15.114:6789/0 conn(0x7fa93c0af4d0 legacy=0x7fa93c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93bfff700 1 -- 172.21.15.114:0/1597950124 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0ae710 con 0x7fa93c06c710 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.396+0000 7fa93a7fc700 1 -- 172.21.15.114:0/1597950124 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576929054 0 0) 0x7fa93c0ae430 con 0x7fa93c0af4d0 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa93bfff700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c0afb90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa934009dc0 tx=0x7fa93400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8046a0f2b8c8ecb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa93a7fc700 1 -- 172.21.15.114:0/1597950124 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400e070 con 0x7fa93c06c710 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa93a7fc700 1 -- 172.21.15.114:0/1597950124 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93400bb90 con 0x7fa93c06c710 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa93a7fc700 1 -- 172.21.15.114:0/1597950124 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934012430 con 0x7fa93c06c710 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 -- 172.21.15.114:0/1597950124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 msgr2=0x7fa93c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c0afb90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa934009dc0 tx=0x7fa93400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 -- 172.21.15.114:0/1597950124 shutdown_connections 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c15e080 0x7fa93c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 --2- 172.21.15.114:0/1597950124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c0afb90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 -- 172.21.15.114:0/1597950124 >> 172.21.15.114:0/1597950124 conn(0x7fa93c071b90 msgr2=0x7fa93c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 -- 172.21.15.114:0/1597950124 shutdown_connections 2023-11-24T06:27:41.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.397+0000 7fa94212b700 1 -- 172.21.15.114:0/1597950124 wait complete. 2023-11-24T06:27:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.398+0000 7fa94212b700 1 Processor -- start 2023-11-24T06:27:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 -- start start 2023-11-24T06:27:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c0af4d0 0x7fa93c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa93c15e080 0x7fa93c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c157400 con 0x7fa93c06c710 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c1576e0 con 0x7fa93c15e080 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.399+0000 7fa94212b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c1579c0 con 0x7fa93c0af4d0 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41996/0 (socket says 172.21.15.114:41996) 2023-11-24T06:27:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 -- 172.21.15.114:0/63989295 learned_addr learned my addr 172.21.15.114:0/63989295 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 -- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c0af4d0 msgr2=0x7fa93c152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c0af4d0 0x7fa93c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 -- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa93c15e080 msgr2=0x7fa93c152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa93c15e080 0x7fa93c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 -- 172.21.15.114:0/63989295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934009a70 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa93b7fe700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa93000d800 tx=0x7fa93000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300125f0 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930012750 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c157ca0 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930012b00 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.400+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c158260 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.401+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c158420 con 0x7fa93c06c710 2023-11-24T06:27:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.402+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa930012c60 con 0x7fa93c06c710 2023-11-24T06:27:41.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.402+0000 7fa923fff700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 0x7fa92806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.402+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9300899e0 con 0x7fa93c06c710 2023-11-24T06:27:41.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.402+0000 7fa93affd700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 0x7fa92806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.408 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.402+0000 7fa93affd700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 0x7fa92806d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0011e0 tx=0x7fa92c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.406+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa930054ad0 con 0x7fa93c06c710 2023-11-24T06:27:41.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.599+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa93c0753f0 con 0x7fa93c06c710 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.599+0000 7fa923fff700 1 -- 172.21.15.114:0/63989295 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa9300580f0 con 0x7fa93c06c710 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2023-11-24T06:27:41.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:41.604 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:27:41.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 msgr2=0x7fa92806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 0x7fa92806d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0011e0 tx=0x7fa92c0079b0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 msgr2=0x7fa93c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa93000d800 tx=0x7fa93000dbc0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 shutdown_connections 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa92806b080 0x7fa92806d540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa93c15e080 0x7fa93c152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa93c06c710 0x7fa93c15bd70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.601+0000 7fa94212b700 1 --2- 172.21.15.114:0/63989295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa93c0af4d0 0x7fa93c152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.602+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 >> 172.21.15.114:0/63989295 conn(0x7fa93c071b90 msgr2=0x7fa93c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.602+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 shutdown_connections 2023-11-24T06:27:41.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.602+0000 7fa94212b700 1 -- 172.21.15.114:0/63989295 wait complete. 2023-11-24T06:27:41.617 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:27:41.617 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:27:41.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.738+0000 7f62a12b3700 1 Processor -- start 2023-11-24T06:27:41.741 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.738+0000 7f62a12b3700 1 -- start start 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f62a12b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f62a12b3700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 0x7f629c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f62a12b3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c0cf4a0 con 0x7f629c0cf050 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f62a12b3700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c15e530 con 0x7f629c161bb0 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f62a12b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0d21a0 con 0x7f629c15e150 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629a7fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 0x7f629c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36546/0 (socket says 172.21.15.114:36546) 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 -- 172.21.15.114:0/3250929863 learned_addr learned my addr 172.21.15.114:0/3250929863 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:41.744 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629b7fe700 1 --1- 172.21.15.114:0/3250929863 >> v1:172.21.15.172:6789/0 conn(0x7f629c0cf050 0x7f629c0b58c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:51310/0 (socket says 172.21.15.114:51310) 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 -- 172.21.15.114:0/3250929863 >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 msgr2=0x7f629c0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 --2- 172.21.15.114:0/3250929863 >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 0x7f629c0ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 -- 172.21.15.114:0/3250929863 >> v1:172.21.15.172:6789/0 conn(0x7f629c0cf050 legacy=0x7f629c0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629affd700 1 -- 172.21.15.114:0/3250929863 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c162390 con 0x7f629c15e150 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.739+0000 7f629a7fc700 1 --2- 172.21.15.114:0/3250929863 >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 0x7f629c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f629affd700 1 --2- 172.21.15.114:0/3250929863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62900098f0 tx=0x7f6290009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8ec25438b6e0e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f6299ffb700 1 -- 172.21.15.114:0/3250929863 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290011830 con 0x7f629c15e150 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f6299ffb700 1 -- 172.21.15.114:0/3250929863 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6290011990 con 0x7f629c15e150 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f6299ffb700 1 -- 172.21.15.114:0/3250929863 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290011d20 con 0x7f629c15e150 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 -- 172.21.15.114:0/3250929863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 msgr2=0x7f629c0b9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 --2- 172.21.15.114:0/3250929863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f62900098f0 tx=0x7f6290009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:41.745 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 -- 172.21.15.114:0/3250929863 shutdown_connections 2023-11-24T06:27:41.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 --2- 172.21.15.114:0/3250929863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c0b9a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 --2- 172.21.15.114:0/3250929863 >> v2:172.21.15.172:3300/0 conn(0x7f629c161bb0 0x7f629c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 -- 172.21.15.114:0/3250929863 >> 172.21.15.114:0/3250929863 conn(0x7f629c06b620 msgr2=0x7f629c071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:41.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.740+0000 7f62a12b3700 1 -- 172.21.15.114:0/3250929863 shutdown_connections 2023-11-24T06:27:41.746 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.741+0000 7f62a12b3700 1 -- 172.21.15.114:0/3250929863 wait complete. 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.742+0000 7f62a12b3700 1 Processor -- start 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.742+0000 7f62a12b3700 1 -- start start 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f629c0cf050 0x7f629c14bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c14b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c161540 con 0x7f629c161bb0 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c158ea0 con 0x7f629c0cf050 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0cb0b0 con 0x7f629c15e150 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.748 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42012/0 (socket says 172.21.15.114:42012) 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 -- 172.21.15.114:0/20437630 learned_addr learned my addr 172.21.15.114:0/20437630 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629a7fc700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c14b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629affd700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f629c0cf050 0x7f629c14bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 -- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 msgr2=0x7f629c14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c14b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 -- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f629c0cf050 msgr2=0x7f629c14bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f629c0cf050 0x7f629c14bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 -- 172.21.15.114:0/20437630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0094a0 con 0x7f629c161bb0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f629b7fe700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f629400cbc0 tx=0x7f629400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400a9e0 con 0x7f629c161bb0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400ab40 con 0x7f629c161bb0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6290009550 con 0x7f629c161bb0 2023-11-24T06:27:41.749 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294007070 con 0x7f629c161bb0 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.743+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c0cb670 con 0x7f629c161bb0 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.744+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0cbcc0 con 0x7f629c161bb0 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.745+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f62940071d0 con 0x7f629c161bb0 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.745+0000 7f62837fe700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 0x7f628806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.745+0000 7f629affd700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 0x7f628806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:41.750 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.745+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6294088fe0 con 0x7f629c161bb0 2023-11-24T06:27:41.751 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.746+0000 7f629affd700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 0x7f628806d570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6290014040 tx=0x7f6290008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:41.753 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.750+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6294054180 con 0x7f629c161bb0 2023-11-24T06:27:41.947 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:41.944+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f629c0cf4a0 con 0x7f629c161bb0 2023-11-24T06:27:42.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.866+0000 7f62837fe700 1 -- 172.21.15.114:0/20437630 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7f629c0cf4a0 con 0x7f629c161bb0 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 msgr2=0x7f628806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 0x7f628806d570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6290014040 tx=0x7f6290008030 comp rx=0 tx=0).stop 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 msgr2=0x7f629c0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f629400cbc0 tx=0x7f629400ced0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 shutdown_connections 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f628806b0b0 0x7f628806d570 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f629c161bb0 0x7f629c0b5470 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f629c0cf050 0x7f629c14bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 --2- 172.21.15.114:0/20437630 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f629c15e150 0x7f629c14b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 >> 172.21.15.114:0/20437630 conn(0x7f629c06b620 msgr2=0x7f629c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:42.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 shutdown_connections 2023-11-24T06:27:42.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:42.869+0000 7f62a12b3700 1 -- 172.21.15.114:0/20437630 wait complete. 2023-11-24T06:27:42.874 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2023-11-24T06:27:42.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:27:43.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.019+0000 7f0ed21f0700 1 Processor -- start 2023-11-24T06:27:43.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.019+0000 7f0ed21f0700 1 -- start start 2023-11-24T06:27:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ed21f0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ecc15b370 0x7f0ecc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ed21f0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0ecc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ed21f0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0ae5e0 con 0x7f0ecc1587f0 2023-11-24T06:27:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ed21f0700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b4630 con 0x7f0ecc15b370 2023-11-24T06:27:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ed21f0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc15be30 con 0x7f0ecc0b3c60 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ecb7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0ecc1587f0 0x7f0ecc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57724/0 (socket says 172.21.15.114:57724) 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ecb7fe700 1 -- 172.21.15.114:0/736777513 learned_addr learned my addr 172.21.15.114:0/736777513 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ecaffd700 1 --2- 172.21.15.114:0/736777513 >> v2:172.21.15.172:3300/0 conn(0x7f0ecc15b370 0x7f0ecc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0ecbfff700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0ecc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.020+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285719763 0 0) 0x7f0ecc0ae5e0 con 0x7f0ecc1587f0 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb40035b0 con 0x7f0ecc1587f0 2023-11-24T06:27:43.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901991299 0 0) 0x7f0eb40035b0 con 0x7f0ecc1587f0 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 msgr2=0x7f0ecc0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0ecc0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 >> v2:172.21.15.172:3300/0 conn(0x7f0ecc15b370 msgr2=0x7f0ecc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 --2- 172.21.15.114:0/736777513 >> v2:172.21.15.172:3300/0 conn(0x7f0ecc15b370 0x7f0ecc15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc15c110 con 0x7f0ecc1587f0 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0ec0001670 con 0x7f0ecc1587f0 2023-11-24T06:27:43.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 >> v1:172.21.15.114:6789/0 conn(0x7f0ecc1587f0 legacy=0x7f0ecc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0eb4003a00 0x7f0eb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0eb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4004d90 0x7f0eb4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b4630 con 0x7f0eb4003a00 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc15be30 con 0x7f0eb4004d90 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb400ba10 con 0x7f0ecc0b3c60 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0eb4003a00 0x7f0eb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecbfff700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4004d90 0x7f0eb4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecaffd700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0eb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 -- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 msgr2=0x7f0eb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc0b3c60 0x7f0eb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 -- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4004d90 msgr2=0x7f0eb4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4004d90 0x7f0eb4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 -- 172.21.15.114:0/736777513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40035b0 con 0x7f0eb4003a00 2023-11-24T06:27:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.021+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0eb4003a00 0x7f0eb4003e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c240 tx=0x7f0eb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374c2e8677d50266 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0007c10 con 0x7f0eb4003a00 2023-11-24T06:27:43.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec000ce00 con 0x7f0eb4003a00 2023-11-24T06:27:43.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0eca7fc700 1 -- 172.21.15.114:0/736777513 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0005ba0 con 0x7f0eb4003a00 2023-11-24T06:27:43.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 -- 172.21.15.114:0/736777513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0eb4003a00 msgr2=0x7f0eb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/736777513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0eb4003a00 0x7f0eb4003e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000c240 tx=0x7f0eb4005880 comp rx=0 tx=0).stop 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ecbfff700 1 -- 172.21.15.114:0/736777513 reap_dead start 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 -- 172.21.15.114:0/736777513 shutdown_connections 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 -- 172.21.15.114:0/736777513 >> 172.21.15.114:0/736777513 conn(0x7f0ecc0d0bf0 msgr2=0x7f0ecc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 -- 172.21.15.114:0/736777513 shutdown_connections 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.022+0000 7f0ed21f0700 1 -- 172.21.15.114:0/736777513 wait complete. 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.023+0000 7f0ed21f0700 1 Processor -- start 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 -- start start 2023-11-24T06:27:43.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 0x7f0ecc1421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 0x7f0ecc0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b1b60 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc142710 con 0x7f0eb4003a00 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.024+0000 7f0ed21f0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc143350 con 0x7f0ecc09bfe0 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42032/0 (socket says 172.21.15.114:42032) 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 -- 172.21.15.114:0/3404211777 learned_addr learned my addr 172.21.15.114:0/3404211777 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecaffd700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 0x7f0ecc0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 0x7f0ecc1421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 -- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 msgr2=0x7f0ecc0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 0x7f0ecc0b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 -- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 msgr2=0x7f0ecc1421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 0x7f0ecc1421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 -- 172.21.15.114:0/3404211777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc145900 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 0x7f0ecc1421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecaffd700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 0x7f0ecc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ecbfff700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0b4a50 tx=0x7f0ec4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400e040 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400f040 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc142a20 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400fdb0 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.025+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc142fe0 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.026+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc068fb0 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.027+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0ec4022460 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.027+0000 7f0eb3fff700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 0x7f0eb806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.027+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0ec408aaf0 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.027+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 0x7f0eb806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.028+0000 7f0ecb7fe700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 0x7f0eb806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0015040 tx=0x7f0ec0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.031+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ec4055a40 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.227 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.224+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0ecc143500 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.868+0000 7f0eb3fff700 1 -- 172.21.15.114:0/3404211777 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0ecc143500 con 0x7f0ecc0b16b0 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 msgr2=0x7f0eb806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 0x7f0eb806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0015040 tx=0x7f0ec0007000 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 msgr2=0x7f0ecc144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0b4a50 tx=0x7f0ec4011f80 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 shutdown_connections 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0eb806b080 0x7f0eb806d540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ecc0b16b0 0x7f0ecc144bd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb4003a00 0x7f0ecc1421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 --2- 172.21.15.114:0/3404211777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ecc09bfe0 0x7f0ecc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.871+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 >> 172.21.15.114:0/3404211777 conn(0x7f0ecc0d0bf0 msgr2=0x7f0ecc15c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.872+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 shutdown_connections 2023-11-24T06:27:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:43.873+0000 7f0ed21f0700 1 -- 172.21.15.114:0/3404211777 wait complete. 2023-11-24T06:27:43.884 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2023-11-24T06:27:43.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:27:44.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.024+0000 7f29707eb700 1 Processor -- start 2023-11-24T06:27:44.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.024+0000 7f29707eb700 1 -- start start 2023-11-24T06:27:44.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 0x7f29680b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29680afc90 0x7f2968073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680744d0 0x7f296806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29680b4500 con 0x7f29680afc90 2023-11-24T06:27:44.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2968074960 con 0x7f296815e160 2023-11-24T06:27:44.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f29707eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29680b3c60 con 0x7f29680744d0 2023-11-24T06:27:44.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.025+0000 7f296dd86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29680afc90 0x7f2968073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f29680afc90 0x7f2968073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42036/0 (socket says 172.21.15.114:42036) 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 -- 172.21.15.114:0/3546163931 learned_addr learned my addr 172.21.15.114:0/3546163931 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296ed88700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680744d0 0x7f296806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 -- 172.21.15.114:0/3546163931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680744d0 msgr2=0x7f296806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296e587700 1 --2- 172.21.15.114:0/3546163931 >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 0x7f29680b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680744d0 0x7f296806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 -- 172.21.15.114:0/3546163931 >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 msgr2=0x7f29680b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 --2- 172.21.15.114:0/3546163931 >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 0x7f29680b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 -- 172.21.15.114:0/3546163931 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680b58c0 con 0x7f29680afc90 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296e587700 1 --2- 172.21.15.114:0/3546163931 >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 0x7f29680b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296dd86700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29680afc90 0x7f2968073f90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f296809efa0 tx=0x7f2964009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbb59a60d09314a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296d585700 1 -- 172.21.15.114:0/3546163931 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400b870 con 0x7f29680afc90 2023-11-24T06:27:44.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296d585700 1 -- 172.21.15.114:0/3546163931 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296400b9d0 con 0x7f29680afc90 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.026+0000 7f296d585700 1 -- 172.21.15.114:0/3546163931 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400bd60 con 0x7f29680afc90 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 -- 172.21.15.114:0/3546163931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29680afc90 msgr2=0x7f2968073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29680afc90 0x7f2968073f90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f296809efa0 tx=0x7f2964009bd0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 -- 172.21.15.114:0/3546163931 shutdown_connections 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680744d0 0x7f296806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 --2- 172.21.15.114:0/3546163931 >> v2:172.21.15.172:3300/0 conn(0x7f296815e160 0x7f29680b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 --2- 172.21.15.114:0/3546163931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29680afc90 0x7f2968073f90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 -- 172.21.15.114:0/3546163931 >> 172.21.15.114:0/3546163931 conn(0x7f29680d0bf0 msgr2=0x7f29680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 -- 172.21.15.114:0/3546163931 shutdown_connections 2023-11-24T06:27:44.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.027+0000 7f29707eb700 1 -- 172.21.15.114:0/3546163931 wait complete. 2023-11-24T06:27:44.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.028+0000 7f29707eb700 1 Processor -- start 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.028+0000 7f29707eb700 1 -- start start 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29680744d0 0x7f296815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680afc90 0x7f2968160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296815ebd0 con 0x7f296815e160 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296815eeb0 con 0x7f29680744d0 2023-11-24T06:27:44.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f29707eb700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296815f190 con 0x7f29680afc90 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f296ed88700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f296ed88700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42044/0 (socket says 172.21.15.114:42044) 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.029+0000 7f296ed88700 1 -- 172.21.15.114:0/3472994937 learned_addr learned my addr 172.21.15.114:0/3472994937 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 -- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680afc90 msgr2=0x7f2968160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680afc90 0x7f2968160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 -- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29680744d0 msgr2=0x7f296815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29680744d0 0x7f296815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 -- 172.21.15.114:0/3472994937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964009550 con 0x7f296815e160 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f296ed88700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f296000cd70 tx=0x7f296000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960003a20 con 0x7f296815e160 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960003b80 con 0x7f296815e160 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296815f470 con 0x7f296815e160 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960004020 con 0x7f296815e160 2023-11-24T06:27:44.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.030+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2968161360 con 0x7f296815e160 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.031+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2968068fb0 con 0x7f296815e160 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.031+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2960004180 con 0x7f296815e160 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.032+0000 7f295effd700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 0x7f294c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.032+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2960012070 con 0x7f296815e160 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.032+0000 7f296e587700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 0x7f294c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.046 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.032+0000 7f296e587700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 0x7f294c06d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2958006ba0 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.047 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.036+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2960054f60 con 0x7f296815e160 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:44.271+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f296806c140 con 0x7f296815e160 2023-11-24T06:27:45.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.179+0000 7f295effd700 1 -- 172.21.15.114:0/3472994937 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2960058580 con 0x7f296815e160 2023-11-24T06:27:45.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 msgr2=0x7f294c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 0x7f294c06d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2958006ba0 tx=0x7f2958008040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 msgr2=0x7f2968151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f296000cd70 tx=0x7f296000a570 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 shutdown_connections 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f294c06b080 0x7f294c06d540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296815e160 0x7f2968151960 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29680744d0 0x7f296815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 --2- 172.21.15.114:0/3472994937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29680afc90 0x7f2968160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 >> 172.21.15.114:0/3472994937 conn(0x7f29680d0bf0 msgr2=0x7f296815d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 shutdown_connections 2023-11-24T06:27:45.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.199+0000 7f29707eb700 1 -- 172.21.15.114:0/3472994937 wait complete. 2023-11-24T06:27:45.204 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T06:27:45.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:45.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.341+0000 7f9268129700 1 Processor -- start 2023-11-24T06:27:45.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.341+0000 7f9268129700 1 -- start start 2023-11-24T06:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f9268129700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92600b2050 0x7f926015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f9268129700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f92600b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f9268129700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926015f910 con 0x7f9260071360 2023-11-24T06:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f9268129700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926015d290 con 0x7f92600b2050 2023-11-24T06:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f9268129700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b2970 con 0x7f92600cccd0 2023-11-24T06:27:45.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f92666c6700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9260071360 0x7f92600717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57772/0 (socket says 172.21.15.114:57772) 2023-11-24T06:27:45.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.342+0000 7f92666c6700 1 -- 172.21.15.114:0/3080794642 learned_addr learned my addr 172.21.15.114:0/3080794642 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:45.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92656c4700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f92600b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9265ec5700 1 --2- 172.21.15.114:0/3080794642 >> v2:172.21.15.172:3300/0 conn(0x7f92600b2050 0x7f926015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262895703 0 0) 0x7f926015f910 con 0x7f9260071360 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f924c0035b0 con 0x7f9260071360 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329524317 0 0) 0x7f924c0035b0 con 0x7f9260071360 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 msgr2=0x7f92600b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f92600b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 >> v2:172.21.15.172:3300/0 conn(0x7f92600b2050 msgr2=0x7f926015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 --2- 172.21.15.114:0/3080794642 >> v2:172.21.15.172:3300/0 conn(0x7f92600b2050 0x7f926015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600cd1b0 con 0x7f9260071360 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f92580019f0 con 0x7f9260071360 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 >> v1:172.21.15.114:6789/0 conn(0x7f9260071360 legacy=0x7f92600717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f924c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c003f80 0x7f924c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f924c004d90 0x7f924c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926015d290 con 0x7f924c004d90 2023-11-24T06:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b2970 con 0x7f924c003f80 2023-11-24T06:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c00ba60 con 0x7f92600cccd0 2023-11-24T06:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9265ec5700 1 --2- 172.21.15.114:0/3080794642 >> v2:172.21.15.172:3300/0 conn(0x7f92600b2050 0x7f926015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f924c004d90 0x7f924c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92656c4700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c003f80 0x7f924c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f9265ec5700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f924c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 -- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 msgr2=0x7f924c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92600cccd0 0x7f924c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 -- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c003f80 msgr2=0x7f924c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.359 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c003f80 0x7f924c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.343+0000 7f92666c6700 1 -- 172.21.15.114:0/3080794642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0035b0 con 0x7f924c004d90 2023-11-24T06:27:45.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f92666c6700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f924c004d90 0x7f924c005210 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f925800a890 tx=0x7f924c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d85922602676992 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258012600 con 0x7f924c004d90 2023-11-24T06:27:45.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9258012760 con 0x7f924c004d90 2023-11-24T06:27:45.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9264ec3700 1 -- 172.21.15.114:0/3080794642 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258012b10 con 0x7f924c004d90 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 -- 172.21.15.114:0/3080794642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f924c004d90 msgr2=0x7f924c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 --2- 172.21.15.114:0/3080794642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f924c004d90 0x7f924c005210 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f925800a890 tx=0x7f924c005880 comp rx=0 tx=0).stop 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f92666c6700 1 -- 172.21.15.114:0/3080794642 reap_dead start 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 -- 172.21.15.114:0/3080794642 shutdown_connections 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 -- 172.21.15.114:0/3080794642 >> 172.21.15.114:0/3080794642 conn(0x7f92600743b0 msgr2=0x7f926006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 -- 172.21.15.114:0/3080794642 shutdown_connections 2023-11-24T06:27:45.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.344+0000 7f9268129700 1 -- 172.21.15.114:0/3080794642 wait complete. 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.345+0000 7f9268129700 1 Processor -- start 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 -- start start 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92601f1cf0 0x7f926014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9260152120 0x7f9260146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92601525a0 con 0x7f926013b870 2023-11-24T06:27:45.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92601f5880 con 0x7f9260152120 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9268129700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926014bcf0 con 0x7f92601f1cf0 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f92656c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f92656c4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42074/0 (socket says 172.21.15.114:42074) 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f92656c4700 1 -- 172.21.15.114:0/4140943512 learned_addr learned my addr 172.21.15.114:0/4140943512 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f92666c6700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9260152120 0x7f9260146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.363 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.346+0000 7f9265ec5700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92601f1cf0 0x7f926014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 -- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92601f1cf0 msgr2=0x7f926014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92601f1cf0 0x7f926014b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 -- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9260152120 msgr2=0x7f9260146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9260152120 0x7f9260146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 -- 172.21.15.114:0/4140943512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92601f5ad0 con 0x7f926013b870 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92656c4700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f925c011ad0 tx=0x7f925c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.364 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00f8e0 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c00fa40 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00fdf0 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92601f5db0 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.347+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92601f6370 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.348+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9260067f60 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.348+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f925c014440 con 0x7f926013b870 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.349+0000 7f92567fc700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 0x7f924406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.365 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.349+0000 7f9265ec5700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 0x7f924406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.349+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f925c089f60 con 0x7f926013b870 2023-11-24T06:27:45.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.349+0000 7f9265ec5700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 0x7f924406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d80 tx=0x7f9250008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.366 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.353+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f925c054db0 con 0x7f926013b870 2023-11-24T06:27:45.548 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.545+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9260062740 con 0x7f926013b870 2023-11-24T06:27:45.550 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.546+0000 7f92567fc700 1 -- 172.21.15.114:0/4140943512 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16614 (secure 0 0 0) 0x7f925c022020 con 0x7f926013b870 2023-11-24T06:27:45.550 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:45.550 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":57,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:44.273655+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 msgr2=0x7f924406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 0x7f924406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9250009d80 tx=0x7f9250008040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 msgr2=0x7f9260151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f925c011ad0 tx=0x7f925c011de0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 shutdown_connections 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f924406b080 0x7f924406d540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9260152120 0x7f9260146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92601f1cf0 0x7f926014b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 --2- 172.21.15.114:0/4140943512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f926013b870 0x7f9260151be0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 >> 172.21.15.114:0/4140943512 conn(0x7f92600743b0 msgr2=0x7f92600cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 shutdown_connections 2023-11-24T06:27:45.553 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.548+0000 7f9268129700 1 -- 172.21.15.114:0/4140943512 wait complete. 2023-11-24T06:27:45.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.680+0000 7fbc1e11e700 1 Processor -- start 2023-11-24T06:27:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.680+0000 7fbc1e11e700 1 -- start start 2023-11-24T06:27:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc1e11e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc1815be00 0x7fbc180cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc1e11e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc180b00e0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc1e11e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc180af200 con 0x7fbc180d0040 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc1e11e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc180af4d0 con 0x7fbc1815be00 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc177fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc1815be00 0x7fbc180cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc16ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbc1815c1e0 0x7fbc180aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57790/0 (socket says 172.21.15.114:57790) 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.681+0000 7fbc16ffd700 1 -- 172.21.15.114:0/3903295745 learned_addr learned my addr 172.21.15.114:0/3903295745 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc177fe700 1 -- 172.21.15.114:0/3903295745 >> v1:172.21.15.172:6789/0 conn(0x7fbc180d0040 legacy=0x7fbc180b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc177fe700 1 -- 172.21.15.114:0/3903295745 >> v1:172.21.15.114:6789/0 conn(0x7fbc1815c1e0 legacy=0x7fbc180aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc177fe700 1 -- 172.21.15.114:0/3903295745 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc180af7b0 con 0x7fbc1815be00 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc167fc700 1 -- 172.21.15.114:0/3903295745 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430422375 0 0) 0x7fbc180b00e0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc177fe700 1 --2- 172.21.15.114:0/3903295745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815be00 0x7fbc180cc660 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009840 tx=0x7fbc0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970afdbe490b4ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc167fc700 1 -- 172.21.15.114:0/3903295745 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c004230 con 0x7fbc1815be00 2023-11-24T06:27:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc167fc700 1 -- 172.21.15.114:0/3903295745 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c004390 con 0x7fbc1815be00 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc167fc700 1 -- 172.21.15.114:0/3903295745 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0036a0 con 0x7fbc1815be00 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc1e11e700 1 -- 172.21.15.114:0/3903295745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815be00 msgr2=0x7fbc180cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.682+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/3903295745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815be00 0x7fbc180cc660 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009840 tx=0x7fbc0c009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.683+0000 7fbc1e11e700 1 -- 172.21.15.114:0/3903295745 shutdown_connections 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.683+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/3903295745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815be00 0x7fbc180cc660 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.683+0000 7fbc1e11e700 1 -- 172.21.15.114:0/3903295745 >> 172.21.15.114:0/3903295745 conn(0x7fbc1809aa00 msgr2=0x7fbc1809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.683+0000 7fbc1e11e700 1 -- 172.21.15.114:0/3903295745 shutdown_connections 2023-11-24T06:27:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.683+0000 7fbc1e11e700 1 -- 172.21.15.114:0/3903295745 wait complete. 2023-11-24T06:27:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.684+0000 7fbc1e11e700 1 Processor -- start 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.684+0000 7fbc1e11e700 1 -- start start 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 0x7fbc181f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 0x7fbc181f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1815ec60 con 0x7fbc1815c1e0 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1815e330 con 0x7fbc1815be00 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1815e610 con 0x7fbc180d0040 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42090/0 (socket says 172.21.15.114:42090) 2023-11-24T06:27:45.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 -- 172.21.15.114:0/2882028566 learned_addr learned my addr 172.21.15.114:0/2882028566 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc177fe700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 0x7fbc181f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc16ffd700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 0x7fbc181f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 -- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 msgr2=0x7fbc181f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 0x7fbc181f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 -- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 msgr2=0x7fbc181f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 0x7fbc181f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 -- 172.21.15.114:0/2882028566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0094a0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc16ffd700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 0x7fbc181f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc177fe700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 0x7fbc181f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc17fff700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000cc10 tx=0x7fbc1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000abb0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1000ad10 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10012660 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1815e8f0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.685+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc181f7920 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.686+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1815ee10 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.687+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbc100127c0 con 0x7fbc1815c1e0 2023-11-24T06:27:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.687+0000 7fbbfffff700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 0x7fbc0406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.687+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc10089f90 con 0x7fbc1815c1e0 2023-11-24T06:27:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.687+0000 7fbc177fe700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 0x7fbc0406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.688+0000 7fbc177fe700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 0x7fbc0406d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.692+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc100bce90 con 0x7fbc1815c1e0 2023-11-24T06:27:45.887 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.884+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc181f7920 con 0x7fbc1815c1e0 2023-11-24T06:27:45.888 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.885+0000 7fbbfffff700 1 -- 172.21.15.114:0/2882028566 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16614 (secure 0 0 0) 0x7fbc181f7920 con 0x7fbc1815c1e0 2023-11-24T06:27:45.889 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:45.889 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":57,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:44.273655+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 msgr2=0x7fbc0406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 0x7fbc0406d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c01c040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 msgr2=0x7fbc1814d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000cc10 tx=0x7fbc1000a060 comp rx=0 tx=0).stop 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 shutdown_connections 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbc0406b080 0x7fbc0406d540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc1815c1e0 0x7fbc1814d7f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc180d0040 0x7fbc181f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 --2- 172.21.15.114:0/2882028566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc1815be00 0x7fbc181f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 >> 172.21.15.114:0/2882028566 conn(0x7fbc1809aa00 msgr2=0x7fbc18061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 shutdown_connections 2023-11-24T06:27:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:45.887+0000 7fbc1e11e700 1 -- 172.21.15.114:0/2882028566 wait complete. 2023-11-24T06:27:45.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:27:46.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.023+0000 7fc7e2d0d700 1 Processor -- start 2023-11-24T06:27:46.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.023+0000 7fc7e2d0d700 1 -- start start 2023-11-24T06:27:46.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e2d0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7dc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e2d0d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc0ae5e0 con 0x7fc7dc1587f0 2023-11-24T06:27:46.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e2d0d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc15be30 con 0x7fc7dc15b370 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e2d0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc15c110 con 0x7fc7dc0b4370 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e0aa9700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc7dc1587f0 0x7fc7dc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57802/0 (socket says 172.21.15.114:57802) 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e0aa9700 1 -- 172.21.15.114:0/2836392252 learned_addr learned my addr 172.21.15.114:0/2836392252 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7dc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487167550 0 0) 0x7fc7dc0ae5e0 con 0x7fc7dc1587f0 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7cc0035b0 con 0x7fc7dc1587f0 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175738269 0 0) 0x7fc7cc0035b0 con 0x7fc7dc1587f0 2023-11-24T06:27:46.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 msgr2=0x7fc7dc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7dc15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 >> v1:172.21.15.172:6789/0 conn(0x7fc7dc15b370 legacy=0x7fc7dc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc0ae860 con 0x7fc7dc1587f0 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fc7d0001670 con 0x7fc7dc1587f0 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 >> v1:172.21.15.114:6789/0 conn(0x7fc7dc1587f0 legacy=0x7fc7dc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.024+0000 7fc7db7fe700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7cc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7cc003f70 0x7fc7cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7cc004d90 0x7fc7cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc15c110 con 0x7fc7cc004d90 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc00ba70 con 0x7fc7cc003f70 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc00de40 con 0x7fc7dc0b4370 2023-11-24T06:27:46.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7cc004d90 0x7fc7cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e0aa9700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7cc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7dbfff700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7cc003f70 0x7fc7cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 -- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 msgr2=0x7fc7cc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc0b4370 0x7fc7cc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 -- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7cc003f70 msgr2=0x7fc7cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7cc003f70 0x7fc7cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 -- 172.21.15.114:0/2836392252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0035b0 con 0x7fc7cc004d90 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7dbfff700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7cc003f70 0x7fc7cc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7cc004d90 0x7fc7cc005210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009f90 tx=0x7fc7cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e45bc578f9f751 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400ec30 con 0x7fc7cc004d90 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d400f040 con 0x7fc7cc004d90 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.025+0000 7fc7db7fe700 1 -- 172.21.15.114:0/2836392252 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4014480 con 0x7fc7cc004d90 2023-11-24T06:27:46.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/2836392252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7cc004d90 msgr2=0x7fc7cc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/2836392252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7cc004d90 0x7fc7cc005210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009f90 tx=0x7fc7cc005880 comp rx=0 tx=0).stop 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e12aa700 1 -- 172.21.15.114:0/2836392252 reap_dead start 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/2836392252 shutdown_connections 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/2836392252 >> 172.21.15.114:0/2836392252 conn(0x7fc7dc0d0bf0 msgr2=0x7fc7dc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/2836392252 shutdown_connections 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.026+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/2836392252 wait complete. 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.027+0000 7fc7e2d0d700 1 Processor -- start 2023-11-24T06:27:46.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.027+0000 7fc7e2d0d700 1 -- start start 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc155100 0x7fc7dc155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 0x7fc7dc1f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc09bc50 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc09d8e0 con 0x7fc7dc1f2b20 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e2d0d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc155ac0 con 0x7fc7dc155100 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42106/0 (socket says 172.21.15.114:42106) 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 -- 172.21.15.114:0/284106364 learned_addr learned my addr 172.21.15.114:0/284106364 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:46.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 0x7fc7dc1f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 -- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc155100 msgr2=0x7fc7dc155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc155100 0x7fc7dc155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 -- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 msgr2=0x7fc7dc1f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 0x7fc7dc1f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 -- 172.21.15.114:0/284106364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc1f3a80 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e12aa700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 0x7fc7dc1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7e0aa9700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007290 tx=0x7fc7d0007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.028+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000f8d0 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.029+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d000fa50 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.029+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc1f34e0 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.029+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000fe00 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.041 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.029+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc09daf0 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.030+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc09e420 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.030+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc7d001d460 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.030+0000 7fc7d97fa700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 0x7fc7c006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.030+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7d0089a70 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.030+0000 7fc7dbfff700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 0x7fc7c006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.031+0000 7fc7dbfff700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 0x7fc7c006d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8007a10 tx=0x7fc7c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.035+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7d00548a0 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:46.272+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc7dc068fb0 con 0x7fc7dc1eefa0 2023-11-24T06:27:46.718 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:46.721 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:47.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.192+0000 7fc7d97fa700 1 -- 172.21.15.114:0/284106364 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7d0057ec0 con 0x7fc7dc1eefa0 2023-11-24T06:27:47.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 msgr2=0x7fc7c006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 0x7fc7c006d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8007a10 tx=0x7fc7c8009040 comp rx=0 tx=0).stop 2023-11-24T06:27:47.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 msgr2=0x7fc7dc09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007290 tx=0x7fc7d0007650 comp rx=0 tx=0).stop 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 shutdown_connections 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7c006b080 0x7fc7c006d540 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7dc1f2b20 0x7fc7dc1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7dc1eefa0 0x7fc7dc09e5d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 --2- 172.21.15.114:0/284106364 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7dc155100 0x7fc7dc155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 >> 172.21.15.114:0/284106364 conn(0x7fc7dc0d0bf0 msgr2=0x7fc7dc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 shutdown_connections 2023-11-24T06:27:47.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.195+0000 7fc7e2d0d700 1 -- 172.21.15.114:0/284106364 wait complete. 2023-11-24T06:27:47.213 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:27:47.214 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:27:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.342+0000 7fc8b2461700 1 Processor -- start 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.342+0000 7fc8b2461700 1 -- start start 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.343+0000 7fc8b2461700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 0x7fc8ac06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.343+0000 7fc8b2461700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.343+0000 7fc8b2461700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac06ba50 con 0x7fc8ac0b4310 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.343+0000 7fc8b2461700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d1200 con 0x7fc8ac15e080 2023-11-24T06:27:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.343+0000 7fc8b2461700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac15e460 con 0x7fc8ac0ce430 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8abfff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 0x7fc8ac06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36662/0 (socket says 172.21.15.114:36662) 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2931661873 learned_addr learned my addr 172.21.15.114:0/2931661873 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2931661873 >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 msgr2=0x7fc8ac06bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 --2- 172.21.15.114:0/2931661873 >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 0x7fc8ac06bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2931661873 >> v1:172.21.15.172:6789/0 conn(0x7fc8ac0b4310 legacy=0x7fc8ac0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2931661873 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac06b120 con 0x7fc8ac0ce430 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8abfff700 1 --2- 172.21.15.114:0/2931661873 >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 0x7fc8ac06bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8b09fe700 1 --2- 172.21.15.114:0/2931661873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009e50 tx=0x7fc8a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5063ee77dfebafbf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8aaffd700 1 -- 172.21.15.114:0/2931661873 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400e070 con 0x7fc8ac0ce430 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8aaffd700 1 -- 172.21.15.114:0/2931661873 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a400b910 con 0x7fc8ac0ce430 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.344+0000 7fc8aaffd700 1 -- 172.21.15.114:0/2931661873 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400bcc0 con 0x7fc8ac0ce430 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 -- 172.21.15.114:0/2931661873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 msgr2=0x7fc8ac0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2931661873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009e50 tx=0x7fc8a400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 -- 172.21.15.114:0/2931661873 shutdown_connections 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2931661873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0ce430 0x7fc8ac0ce8c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2931661873 >> v2:172.21.15.172:3300/0 conn(0x7fc8ac15e080 0x7fc8ac06bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 -- 172.21.15.114:0/2931661873 >> 172.21.15.114:0/2931661873 conn(0x7fc8ac0b3780 msgr2=0x7fc8ac0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 -- 172.21.15.114:0/2931661873 shutdown_connections 2023-11-24T06:27:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.345+0000 7fc8b2461700 1 -- 172.21.15.114:0/2931661873 wait complete. 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.346+0000 7fc8b2461700 1 Processor -- start 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.346+0000 7fc8b2461700 1 -- start start 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0b4310 0x7fc8ac160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8ac0ce430 0x7fc8ac0b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0cb470 con 0x7fc8ac15e080 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0ca860 con 0x7fc8ac0ce430 2023-11-24T06:27:47.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b2461700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0cab40 con 0x7fc8ac0b4310 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b09fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b09fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42124/0 (socket says 172.21.15.114:42124) 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2298273531 learned_addr learned my addr 172.21.15.114:0/2298273531 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8abfff700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0b4310 0x7fc8ac160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.347+0000 7fc8ab7fe700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8ac0ce430 0x7fc8ac0b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0b4310 msgr2=0x7fc8ac160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0b4310 0x7fc8ac160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8ac0ce430 msgr2=0x7fc8ac0b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8ac0ce430 0x7fc8ac0b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 -- 172.21.15.114:0/2298273531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00094a0 con 0x7fc8ac15e080 2023-11-24T06:27:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b09fe700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0725a0 tx=0x7fc8a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004960 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a40099b0 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a4009b30 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a401c680 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.348+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac0cb100 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.349+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac068fb0 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.349+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc8a401c7e0 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.350+0000 7fc8a8ff9700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 0x7fc89806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.350+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8a408b7d0 con 0x7fc8ac15e080 2023-11-24T06:27:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.350+0000 7fc8abfff700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 0x7fc89806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.350+0000 7fc8abfff700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 0x7fc89806d540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009c60 tx=0x7fc8a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.354+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8a40564e0 con 0x7fc8ac15e080 2023-11-24T06:27:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:47.588+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc8ac0b7ab0 con 0x7fc8ac15e080 2023-11-24T06:27:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.199+0000 7fc8a8ff9700 1 -- 172.21.15.114:0/2298273531 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8a4059b00 con 0x7fc8ac15e080 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 msgr2=0x7fc89806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 0x7fc89806d540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009c60 tx=0x7fc8a0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 msgr2=0x7fc8ac0b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0725a0 tx=0x7fc8a4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 shutdown_connections 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc89806b080 0x7fc89806d540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8ac15e080 0x7fc8ac0b6de0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8ac0b4310 0x7fc8ac160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 --2- 172.21.15.114:0/2298273531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8ac0ce430 0x7fc8ac0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 >> 172.21.15.114:0/2298273531 conn(0x7fc8ac0b3780 msgr2=0x7fc8ac070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 shutdown_connections 2023-11-24T06:27:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.202+0000 7fc8b2461700 1 -- 172.21.15.114:0/2298273531 wait complete. 2023-11-24T06:27:48.219 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:27:48.219 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.337+0000 7fdbbbd64700 1 Processor -- start 2023-11-24T06:27:48.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.337+0000 7fdbbbd64700 1 -- start start 2023-11-24T06:27:48.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbbbd64700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdbb406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbbbd64700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40af950 con 0x7fdbb41446a0 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbbbd64700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40afc30 con 0x7fdbb406c540 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbbbd64700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4144a80 con 0x7fdbb40adf00 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb9b00700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdbb41446a0 0x7fdbb40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57836/0 (socket says 172.21.15.114:57836) 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb9b00700 1 -- 172.21.15.114:0/118944278 learned_addr learned my addr 172.21.15.114:0/118944278 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999286297 0 0) 0x7fdbb40af950 con 0x7fdbb41446a0 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba00035b0 con 0x7fdbb41446a0 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb92ff700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdbb406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279738932 0 0) 0x7fdba00035b0 con 0x7fdbb41446a0 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 msgr2=0x7fdbb406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdbb406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 >> v1:172.21.15.172:6789/0 conn(0x7fdbb406c540 legacy=0x7fdbb40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40aff10 con 0x7fdbb41446a0 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fdbb0001670 con 0x7fdbb41446a0 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 >> v1:172.21.15.114:6789/0 conn(0x7fdbb41446a0 legacy=0x7fdbb40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdba0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0004060 0x7fdba0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdba0004d90 0x7fdba0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4144a80 con 0x7fdba0004d90 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba000bbd0 con 0x7fdba0004060 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.338+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba000bd90 con 0x7fdbb40adf00 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdba0004d90 0x7fdba0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbb9b00700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdba0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbb92ff700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0004060 0x7fdba0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 -- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 msgr2=0x7fdba0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb40adf00 0x7fdba0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 -- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0004060 msgr2=0x7fdba0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0004060 0x7fdba0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 -- 172.21.15.114:0/118944278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00035b0 con 0x7fdba0004d90 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbba301700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdba0004d90 0x7fdba0005210 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c190 tx=0x7fdba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df75f3fbf3f5f08c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac001db0 con 0x7fdba0004d90 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac00ccd0 con 0x7fdba0004d90 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.339+0000 7fdbb8afe700 1 -- 172.21.15.114:0/118944278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac003c40 con 0x7fdba0004d90 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 -- 172.21.15.114:0/118944278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdba0004d90 msgr2=0x7fdba0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/118944278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdba0004d90 0x7fdba0005210 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c190 tx=0x7fdba0005880 comp rx=0 tx=0).stop 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbba301700 1 -- 172.21.15.114:0/118944278 reap_dead start 2023-11-24T06:27:48.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 -- 172.21.15.114:0/118944278 shutdown_connections 2023-11-24T06:27:48.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 -- 172.21.15.114:0/118944278 >> 172.21.15.114:0/118944278 conn(0x7fdbb4071b90 msgr2=0x7fdbb4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 -- 172.21.15.114:0/118944278 shutdown_connections 2023-11-24T06:27:48.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.340+0000 7fdbbbd64700 1 -- 172.21.15.114:0/118944278 wait complete. 2023-11-24T06:27:48.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.341+0000 7fdbbbd64700 1 Processor -- start 2023-11-24T06:27:48.350 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.341+0000 7fdbbbd64700 1 -- start start 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb41f0a20 0x7fdbb413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb40ce5c0 0x7fdbb4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41f3a40 con 0x7fdbb41f0e00 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41f3d20 con 0x7fdbb40ce5c0 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41f4000 con 0x7fdbb41f0a20 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42142/0 (socket says 172.21.15.114:42142) 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 -- 172.21.15.114:0/2857249722 learned_addr learned my addr 172.21.15.114:0/2857249722 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb9b00700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb41f0a20 0x7fdbb413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbba301700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb40ce5c0 0x7fdbb4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 -- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb41f0a20 msgr2=0x7fdbb413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb41f0a20 0x7fdbb413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 -- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb40ce5c0 msgr2=0x7fdbb4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb40ce5c0 0x7fdbb4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 -- 172.21.15.114:0/2857249722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb41f42e0 con 0x7fdbb41f0e00 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbb92ff700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdba4011650 tx=0x7fdba4011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400c070 con 0x7fdbb41f0e00 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4015870 con 0x7fdbb41f0e00 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4015c20 con 0x7fdbb41f0e00 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb41f45c0 con 0x7fdbb41f0e00 2023-11-24T06:27:48.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.342+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb40744d0 con 0x7fdbb41f0e00 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.343+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb40753f0 con 0x7fdbb41f0e00 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.344+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fdba40143e0 con 0x7fdbb41f0e00 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.344+0000 7fdbaa7fc700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 0x7fdb9806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.344+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdba4089b40 con 0x7fdbb41f0e00 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.344+0000 7fdbb9b00700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 0x7fdb9806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.345+0000 7fdbb9b00700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 0x7fdb9806d570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001c90 tx=0x7fdbb0001c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.355 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.349+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba4054860 con 0x7fdbb41f0e00 2023-11-24T06:27:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.580+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbb4067f60 con 0x7fdbb41f0e00 2023-11-24T06:27:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.581+0000 7fdbaa7fc700 1 -- 172.21.15.114:0/2857249722 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fdba4057e80 con 0x7fdbb41f0e00 2023-11-24T06:27:48.584 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:48.585 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:27:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.583+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 msgr2=0x7fdb9806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.583+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 0x7fdb9806d570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001c90 tx=0x7fdbb0001c00 comp rx=0 tx=0).stop 2023-11-24T06:27:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.583+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 msgr2=0x7fdbb40ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.583+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fdba4011650 tx=0x7fdba4011960 comp rx=0 tx=0).stop 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.583+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 shutdown_connections 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fdb9806b0b0 0x7fdb9806d570 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb40ce5c0 0x7fdbb4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdbb41f0a20 0x7fdbb413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 --2- 172.21.15.114:0/2857249722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41f0e00 0x7fdbb40ce080 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 >> 172.21.15.114:0/2857249722 conn(0x7fdbb4071b90 msgr2=0x7fdbb4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 shutdown_connections 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.584+0000 7fdbbbd64700 1 -- 172.21.15.114:0/2857249722 wait complete. 2023-11-24T06:27:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2023-11-24T06:27:48.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:48.719 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.716+0000 7f5f53f62700 1 Processor -- start 2023-11-24T06:27:48.720 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.716+0000 7f5f53f62700 1 -- start start 2023-11-24T06:27:48.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f53f62700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f4c152cc0 0x7f5f4c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f53f62700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 0x7f5f4c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f53f62700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c15c4b0 con 0x7f5f4c0cff20 2023-11-24T06:27:48.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f53f62700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c15c790 con 0x7f5f4c152cc0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f53f62700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c15ca70 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f51cfe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f4c152cc0 0x7f5f4c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f51cfe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5f4c152cc0 0x7f5f4c06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42148/0 (socket says 172.21.15.114:42148) 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f51cfe700 1 -- 172.21.15.114:0/2614596913 learned_addr learned my addr 172.21.15.114:0/2614596913 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.717+0000 7f5f514fd700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 0x7f5f4c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f51cfe700 1 -- 172.21.15.114:0/2614596913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 msgr2=0x7f5f4c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f51cfe700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 0x7f5f4c06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f51cfe700 1 -- 172.21.15.114:0/2614596913 >> v1:172.21.15.172:6789/0 conn(0x7f5f4c0cff20 legacy=0x7f5f4c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f51cfe700 1 -- 172.21.15.114:0/2614596913 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c15cd50 con 0x7f5f4c152cc0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f514fd700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 0x7f5f4c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f51cfe700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c152cc0 0x7f5f4c06c000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0098f0 tx=0x7f5f3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca4839b76ea1043 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f50cfc700 1 -- 172.21.15.114:0/2614596913 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004230 con 0x7f5f4c152cc0 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f50cfc700 1 -- 172.21.15.114:0/2614596913 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c004390 con 0x7f5f4c152cc0 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f50cfc700 1 -- 172.21.15.114:0/2614596913 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0036a0 con 0x7f5f4c152cc0 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 -- 172.21.15.114:0/2614596913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c152cc0 msgr2=0x7f5f4c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c152cc0 0x7f5f4c06c000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0098f0 tx=0x7f5f3c009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 -- 172.21.15.114:0/2614596913 shutdown_connections 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c152cc0 0x7f5f4c06c000 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 --2- 172.21.15.114:0/2614596913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c0b49a0 0x7f5f4c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.718+0000 7f5f53f62700 1 -- 172.21.15.114:0/2614596913 >> 172.21.15.114:0/2614596913 conn(0x7f5f4c071b90 msgr2=0x7f5f4c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.719+0000 7f5f53f62700 1 -- 172.21.15.114:0/2614596913 shutdown_connections 2023-11-24T06:27:48.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.719+0000 7f5f53f62700 1 -- 172.21.15.114:0/2614596913 wait complete. 2023-11-24T06:27:48.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.720+0000 7f5f53f62700 1 Processor -- start 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.720+0000 7f5f53f62700 1 -- start start 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f4c0cff20 0x7f5f4c146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c152cc0 0x7f5f4c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c154ec0 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1569e0 con 0x7f5f4c0cff20 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f53f62700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c150d30 con 0x7f5f4c152cc0 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42154/0 (socket says 172.21.15.114:42154) 2023-11-24T06:27:48.728 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 -- 172.21.15.114:0/4291430085 learned_addr learned my addr 172.21.15.114:0/4291430085 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 -- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c152cc0 msgr2=0x7f5f4c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f524ff700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c152cc0 0x7f5f4c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f514fd700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f4c0cff20 0x7f5f4c146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c152cc0 0x7f5f4c1513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 -- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f4c0cff20 msgr2=0x7f5f4c146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f4c0cff20 0x7f5f4c146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 -- 172.21.15.114:0/4291430085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009550 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.721+0000 7f5f51cfe700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0728e0 tx=0x7f5f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.722+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00b860 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.722+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00b9c0 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.722+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c155110 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.722+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00bd70 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.722+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c1556d0 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.723+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c0753f0 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.723+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5f3c01b440 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.723+0000 7f5f427fc700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 0x7f5f3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.723+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3c08b240 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.724+0000 7f5f514fd700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 0x7f5f3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.724+0000 7f5f514fd700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 0x7f5f3006d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5f48006fd0 tx=0x7f5f48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.728+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f3c056100 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.921 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.917+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f4c156810 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.922 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.919+0000 7f5f427fc700 1 -- 172.21.15.114:0/4291430085 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f5f3c059720 con 0x7f5f4c0b49a0 2023-11-24T06:27:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:48.922 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:48.925 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 msgr2=0x7f5f3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 0x7f5f3006d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5f48006fd0 tx=0x7f5f48008040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 msgr2=0x7f5f4c1507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0728e0 tx=0x7f5f3c004750 comp rx=0 tx=0).stop 2023-11-24T06:27:48.926 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 shutdown_connections 2023-11-24T06:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5f3006b080 0x7f5f3006d540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f4c152cc0 0x7f5f4c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f4c0b49a0 0x7f5f4c1507f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 --2- 172.21.15.114:0/4291430085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f4c0cff20 0x7f5f4c146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 >> 172.21.15.114:0/4291430085 conn(0x7f5f4c071b90 msgr2=0x7f5f4c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 shutdown_connections 2023-11-24T06:27:48.928 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:48.921+0000 7f5f53f62700 1 -- 172.21.15.114:0/4291430085 wait complete. 2023-11-24T06:27:48.937 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:27:48.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.053+0000 7f0a28eb3700 1 Processor -- start 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.054+0000 7f0a28eb3700 1 -- start start 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.054+0000 7f0a28eb3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a240622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.054+0000 7f0a28eb3700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2415c540 con 0x7f0a2415c160 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.054+0000 7f0a28eb3700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240b2ee0 con 0x7f0a240af1e0 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.054+0000 7f0a28eb3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240add40 con 0x7f0a240b04b0 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0a2415c160 0x7f0a240afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57870/0 (socket says 172.21.15.114:57870) 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 -- 172.21.15.114:0/3353895839 learned_addr learned my addr 172.21.15.114:0/3353895839 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a240622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772781906 0 0) 0x7f0a2415c540 con 0x7f0a2415c160 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0c0035b0 con 0x7f0a2415c160 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291147220 0 0) 0x7f0a0c0035b0 con 0x7f0a2415c160 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 msgr2=0x7f0a240622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a240622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 >> v1:172.21.15.172:6789/0 conn(0x7f0a240af1e0 legacy=0x7f0a24165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240ad410 con 0x7f0a2415c160 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0a18001670 con 0x7f0a2415c160 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 >> v1:172.21.15.114:6789/0 conn(0x7f0a2415c160 legacy=0x7f0a240afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c003a30 0x7f0a0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a0c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c004d90 0x7f0a0c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240add40 con 0x7f0a0c003a30 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c00bac0 con 0x7f0a0c004d90 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c00bc80 con 0x7f0a240b04b0 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c003a30 0x7f0a0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a22d9d700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c004d90 0x7f0a0c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a0c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 -- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 msgr2=0x7f0a0c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a0c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.055+0000 7f0a2259c700 1 -- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c004d90 msgr2=0x7f0a0c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2259c700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c004d90 0x7f0a0c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2259c700 1 -- 172.21.15.114:0/3353895839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0035b0 con 0x7f0a0c003a30 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240b04b0 0x7f0a0c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2259c700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c003a30 0x7f0a0c003e50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c210 tx=0x7f0a0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7ce1b4d99a41c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18007c10 con 0x7f0a0c003a30 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1800cdb0 con 0x7f0a0c003a30 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a2159a700 1 -- 172.21.15.114:0/3353895839 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18005ba0 con 0x7f0a0c003a30 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 -- 172.21.15.114:0/3353895839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c003a30 msgr2=0x7f0a0c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/3353895839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c003a30 0x7f0a0c003e50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c210 tx=0x7f0a0c005880 comp rx=0 tx=0).stop 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a22d9d700 1 -- 172.21.15.114:0/3353895839 reap_dead start 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 -- 172.21.15.114:0/3353895839 shutdown_connections 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 -- 172.21.15.114:0/3353895839 >> 172.21.15.114:0/3353895839 conn(0x7f0a240b57b0 msgr2=0x7f0a240b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 -- 172.21.15.114:0/3353895839 shutdown_connections 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.056+0000 7f0a28eb3700 1 -- 172.21.15.114:0/3353895839 wait complete. 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.058+0000 7f0a28eb3700 1 Processor -- start 2023-11-24T06:27:49.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.058+0000 7f0a28eb3700 1 -- start start 2023-11-24T06:27:49.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 0x7f0a2414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240c9640 0x7f0a240c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2414dee0 con 0x7f0a241f7330 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240ca000 con 0x7f0a0c003a30 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240b36a0 con 0x7f0a240c9640 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42176/0 (socket says 172.21.15.114:42176) 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 -- 172.21.15.114:0/4258092467 learned_addr learned my addr 172.21.15.114:0/4258092467 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a22d9d700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240c9640 0x7f0a240c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a2259c700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 0x7f0a2414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 -- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240c9640 msgr2=0x7f0a240c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240c9640 0x7f0a240c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 -- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 msgr2=0x7f0a2414d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 0x7f0a2414d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 -- 172.21.15.114:0/4258092467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240b3980 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a2259c700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 0x7f0a2414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a21d9b700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0a140118d0 tx=0x7f0a14011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14004220 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14004380 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a241f3d50 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140036a0 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.059+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2415d6f0 con 0x7f0a241f7330 2023-11-24T06:27:49.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.060+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240b3b10 con 0x7f0a241f7330 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.061+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0a14003820 con 0x7f0a241f7330 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.061+0000 7f0a12ffd700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.061+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a14089b60 con 0x7f0a241f7330 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.061+0000 7f0a2259c700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.062+0000 7f0a2259c700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a18001930 tx=0x7f0a18007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.070 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.066+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a14054990 con 0x7f0a241f7330 2023-11-24T06:27:49.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.256+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a240ad9d0 con 0x7f0a0406b080 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.264+0000 7f0a12ffd700 1 -- 172.21.15.114:0/4258092467 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283983 (secure 0 0 0) 0x7f0a18022070 con 0x7f0a0406b080 2023-11-24T06:27:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:49.273 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 msgr2=0x7f0a0406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a18001930 tx=0x7f0a18007000 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 msgr2=0x7f0a2415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0a140118d0 tx=0x7f0a14011be0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 shutdown_connections 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a18001930 tx=0x7f0a18007000 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a240c9640 0x7f0a240c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a0c003a30 0x7f0a2414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 --2- 172.21.15.114:0/4258092467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a241f7330 0x7f0a2415cb10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.274 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 >> 172.21.15.114:0/4258092467 conn(0x7f0a240b57b0 msgr2=0x7f0a240b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 shutdown_connections 2023-11-24T06:27:49.275 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.267+0000 7f0a28eb3700 1 -- 172.21.15.114:0/4258092467 wait complete. 2023-11-24T06:27:49.275 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:27:49.283 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-24T06:27:47.752814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222584,"kb_used_data":6396,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752136,"statfs":{"total":773094113280,"available":772866187264,"internally_reserved":0,"allocated":6549504,"data_stored":3388245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881873"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877928+0000","last_change":"2023-11-24T06:27:43.877928+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877928+0000","last_undegraded":"2023-11-24T06:27:43.877928+0000","last_fullsized":"2023-11-24T06:27:43.877928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878401+0000","last_change":"2023-11-24T06:27:43.878401+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878401+0000","last_undegraded":"2023-11-24T06:27:43.878401+0000","last_fullsized":"2023-11-24T06:27:43.878401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876797+0000","last_change":"2023-11-24T06:27:43.876797+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876797+0000","last_undegraded":"2023-11-24T06:27:43.876797+0000","last_fullsized":"2023-11-24T06:27:43.876797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880382+0000","last_change":"2023-11-24T06:27:43.880382+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880382+0000","last_undegraded":"2023-11-24T06:27:43.880382+0000","last_fullsized":"2023-11-24T06:27:43.880382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875033+0000","last_change":"2023-11-24T06:27:43.875033+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875033+0000","last_undegraded":"2023-11-24T06:27:43.875033+0000","last_fullsized":"2023-11-24T06:27:43.875033+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881134+0000","last_change":"2023-11-24T06:27:43.881134+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881134+0000","last_undegraded":"2023-11-24T06:27:43.881134+0000","last_fullsized":"2023-11-24T06:27:43.881134+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884229+0000","last_change":"2023-11-24T06:27:43.884229+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884229+0000","last_undegraded":"2023-11-24T06:27:43.884229+0000","last_fullsized":"2023-11-24T06:27:43.884229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878821+0000","last_change":"2023-11-24T06:27:43.878821+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878821+0000","last_undegraded":"2023-11-24T06:27:43.878821+0000","last_fullsized":"2023-11-24T06:27:43.878821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881743+0000","last_change":"2023-11-24T06:27:43.881743+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881743+0000","last_undegraded":"2023-11-24T06:27:43.881743+0000","last_fullsized":"2023-11-24T06:27:43.881743+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875949+0000","last_change":"2023-11-24T06:27:43.875949+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875949+0000","last_undegraded":"2023-11-24T06:27:43.875949+0000","last_fullsized":"2023-11-24T06:27:43.875949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876187+0000","last_change":"2023-11-24T06:27:43.876187+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876187+0000","last_undegraded":"2023-11-24T06:27:43.876187+0000","last_fullsized":"2023-11-24T06:27:43.876187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878485+0000","last_change":"2023-11-24T06:27:43.878485+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878485+0000","last_undegraded":"2023-11-24T06:27:43.878485+0000","last_fullsized":"2023-11-24T06:27:43.878485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874177+0000","last_change":"2023-11-24T06:27:43.874177+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874177+0000","last_undegraded":"2023-11-24T06:27:43.874177+0000","last_fullsized":"2023-11-24T06:27:43.874177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882087+0000","last_change":"2023-11-24T06:27:43.882087+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882087+0000","last_undegraded":"2023-11-24T06:27:43.882087+0000","last_fullsized":"2023-11-24T06:27:43.882087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883840+0000","last_change":"2023-11-24T06:27:43.883840+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883840+0000","last_undegraded":"2023-11-24T06:27:43.883840+0000","last_fullsized":"2023-11-24T06:27:43.883840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880935+0000","last_change":"2023-11-24T06:27:43.880935+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880935+0000","last_undegraded":"2023-11-24T06:27:43.880935+0000","last_fullsized":"2023-11-24T06:27:43.880935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877635+0000","last_change":"2023-11-24T06:27:43.877635+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877635+0000","last_undegraded":"2023-11-24T06:27:43.877635+0000","last_fullsized":"2023-11-24T06:27:43.877635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875034+0000","last_change":"2023-11-24T06:27:43.875034+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875034+0000","last_undegraded":"2023-11-24T06:27:43.875034+0000","last_fullsized":"2023-11-24T06:27:43.875034+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875072+0000","last_change":"2023-11-24T06:27:43.875072+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875072+0000","last_undegraded":"2023-11-24T06:27:43.875072+0000","last_fullsized":"2023-11-24T06:27:43.875072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876540+0000","last_change":"2023-11-24T06:27:43.876540+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876540+0000","last_undegraded":"2023-11-24T06:27:43.876540+0000","last_fullsized":"2023-11-24T06:27:43.876540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875751+0000","last_change":"2023-11-24T06:27:43.875751+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875751+0000","last_undegraded":"2023-11-24T06:27:43.875751+0000","last_fullsized":"2023-11-24T06:27:43.875751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878158+0000","last_change":"2023-11-24T06:27:43.878158+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878158+0000","last_undegraded":"2023-11-24T06:27:43.878158+0000","last_fullsized":"2023-11-24T06:27:43.878158+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873638+0000","last_change":"2023-11-24T06:27:43.873638+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873638+0000","last_undegraded":"2023-11-24T06:27:43.873638+0000","last_fullsized":"2023-11-24T06:27:43.873638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875332+0000","last_change":"2023-11-24T06:27:43.875332+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875332+0000","last_undegraded":"2023-11-24T06:27:43.875332+0000","last_fullsized":"2023-11-24T06:27:43.875332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874699+0000","last_change":"2023-11-24T06:27:43.874699+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874699+0000","last_undegraded":"2023-11-24T06:27:43.874699+0000","last_fullsized":"2023-11-24T06:27:43.874699+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878658+0000","last_change":"2023-11-24T06:27:43.878658+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878658+0000","last_undegraded":"2023-11-24T06:27:43.878658+0000","last_fullsized":"2023-11-24T06:27:43.878658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884489+0000","last_change":"2023-11-24T06:27:43.884489+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884489+0000","last_undegraded":"2023-11-24T06:27:43.884489+0000","last_fullsized":"2023-11-24T06:27:43.884489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874694+0000","last_change":"2023-11-24T06:27:43.874694+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874694+0000","last_undegraded":"2023-11-24T06:27:43.874694+0000","last_fullsized":"2023-11-24T06:27:43.874694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877093+0000","last_change":"2023-11-24T06:27:43.877093+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877093+0000","last_undegraded":"2023-11-24T06:27:43.877093+0000","last_fullsized":"2023-11-24T06:27:43.877093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874041+0000","last_change":"2023-11-24T06:27:43.874041+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874041+0000","last_undegraded":"2023-11-24T06:27:43.874041+0000","last_fullsized":"2023-11-24T06:27:43.874041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875031+0000","last_change":"2023-11-24T06:27:43.875031+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875031+0000","last_undegraded":"2023-11-24T06:27:43.875031+0000","last_fullsized":"2023-11-24T06:27:43.875031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880505+0000","last_change":"2023-11-24T06:27:43.880505+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880505+0000","last_undegraded":"2023-11-24T06:27:43.880505+0000","last_fullsized":"2023-11-24T06:27:43.880505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882036+0000","last_change":"2023-11-24T06:27:43.882036+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882036+0000","last_undegraded":"2023-11-24T06:27:43.882036+0000","last_fullsized":"2023-11-24T06:27:43.882036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890188+0000","last_change":"2023-11-24T06:27:43.887983+0000","last_active":"2023-11-24T06:27:43.890188+0000","last_peered":"2023-11-24T06:27:43.890188+0000","last_clean":"2023-11-24T06:27:43.890188+0000","last_became_active":"2023-11-24T06:27:43.887687+0000","last_became_peered":"2023-11-24T06:27:43.887687+0000","last_unstale":"2023-11-24T06:27:43.890188+0000","last_undegraded":"2023-11-24T06:27:43.890188+0000","last_fullsized":"2023-11-24T06:27:43.890188+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:23.658351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876827+0000","last_change":"2023-11-24T06:27:43.876827+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876827+0000","last_undegraded":"2023-11-24T06:27:43.876827+0000","last_fullsized":"2023-11-24T06:27:43.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893472+0000","last_change":"2023-11-24T06:27:43.887478+0000","last_active":"2023-11-24T06:27:43.893472+0000","last_peered":"2023-11-24T06:27:43.893472+0000","last_clean":"2023-11-24T06:27:43.893472+0000","last_became_active":"2023-11-24T06:27:43.886904+0000","last_became_peered":"2023-11-24T06:27:43.886904+0000","last_unstale":"2023-11-24T06:27:43.893472+0000","last_undegraded":"2023-11-24T06:27:43.893472+0000","last_fullsized":"2023-11-24T06:27:43.893472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:17.375576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882394+0000","last_change":"2023-11-24T06:27:43.882394+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882394+0000","last_undegraded":"2023-11-24T06:27:43.882394+0000","last_fullsized":"2023-11-24T06:27:43.882394+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.899278+0000","last_change":"2023-11-24T06:27:43.893948+0000","last_active":"2023-11-24T06:27:43.899278+0000","last_peered":"2023-11-24T06:27:43.899278+0000","last_clean":"2023-11-24T06:27:43.899278+0000","last_became_active":"2023-11-24T06:27:43.893661+0000","last_became_peered":"2023-11-24T06:27:43.893661+0000","last_unstale":"2023-11-24T06:27:43.899278+0000","last_undegraded":"2023-11-24T06:27:43.899278+0000","last_fullsized":"2023-11-24T06:27:43.899278+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884086+0000","last_change":"2023-11-24T06:27:43.884086+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884086+0000","last_undegraded":"2023-11-24T06:27:43.884086+0000","last_fullsized":"2023-11-24T06:27:43.884086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887326+0000","last_change":"2023-11-24T06:27:43.884164+0000","last_active":"2023-11-24T06:27:43.887326+0000","last_peered":"2023-11-24T06:27:43.887326+0000","last_clean":"2023-11-24T06:27:43.887326+0000","last_became_active":"2023-11-24T06:27:43.883886+0000","last_became_peered":"2023-11-24T06:27:43.883886+0000","last_unstale":"2023-11-24T06:27:43.887326+0000","last_undegraded":"2023-11-24T06:27:43.887326+0000","last_fullsized":"2023-11-24T06:27:43.887326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:35.939429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877665+0000","last_change":"2023-11-24T06:27:43.877665+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877665+0000","last_undegraded":"2023-11-24T06:27:43.877665+0000","last_fullsized":"2023-11-24T06:27:43.877665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891238+0000","last_change":"2023-11-24T06:27:43.887860+0000","last_active":"2023-11-24T06:27:43.891238+0000","last_peered":"2023-11-24T06:27:43.891238+0000","last_clean":"2023-11-24T06:27:43.891238+0000","last_became_active":"2023-11-24T06:27:43.887522+0000","last_became_peered":"2023-11-24T06:27:43.887522+0000","last_unstale":"2023-11-24T06:27:43.891238+0000","last_undegraded":"2023-11-24T06:27:43.891238+0000","last_fullsized":"2023-11-24T06:27:43.891238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:18.289207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876826+0000","last_change":"2023-11-24T06:27:43.876826+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876826+0000","last_undegraded":"2023-11-24T06:27:43.876826+0000","last_fullsized":"2023-11-24T06:27:43.876826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890657+0000","last_change":"2023-11-24T06:27:43.888671+0000","last_active":"2023-11-24T06:27:43.890657+0000","last_peered":"2023-11-24T06:27:43.890657+0000","last_clean":"2023-11-24T06:27:43.890657+0000","last_became_active":"2023-11-24T06:27:43.888398+0000","last_became_peered":"2023-11-24T06:27:43.888398+0000","last_unstale":"2023-11-24T06:27:43.890657+0000","last_undegraded":"2023-11-24T06:27:43.890657+0000","last_fullsized":"2023-11-24T06:27:43.890657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:40.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873693+0000","last_change":"2023-11-24T06:27:43.873693+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873693+0000","last_undegraded":"2023-11-24T06:27:43.873693+0000","last_fullsized":"2023-11-24T06:27:43.873693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893351+0000","last_change":"2023-11-24T06:27:43.888597+0000","last_active":"2023-11-24T06:27:43.893351+0000","last_peered":"2023-11-24T06:27:43.893351+0000","last_clean":"2023-11-24T06:27:43.893351+0000","last_became_active":"2023-11-24T06:27:43.888055+0000","last_became_peered":"2023-11-24T06:27:43.888055+0000","last_unstale":"2023-11-24T06:27:43.893351+0000","last_undegraded":"2023-11-24T06:27:43.893351+0000","last_fullsized":"2023-11-24T06:27:43.893351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878321+0000","last_change":"2023-11-24T06:27:43.878321+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878321+0000","last_undegraded":"2023-11-24T06:27:43.878321+0000","last_fullsized":"2023-11-24T06:27:43.878321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893456+0000","last_change":"2023-11-24T06:27:43.890543+0000","last_active":"2023-11-24T06:27:43.893456+0000","last_peered":"2023-11-24T06:27:43.893456+0000","last_clean":"2023-11-24T06:27:43.893456+0000","last_became_active":"2023-11-24T06:27:43.890213+0000","last_became_peered":"2023-11-24T06:27:43.890213+0000","last_unstale":"2023-11-24T06:27:43.893456+0000","last_undegraded":"2023-11-24T06:27:43.893456+0000","last_fullsized":"2023-11-24T06:27:43.893456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:21.791667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881850+0000","last_change":"2023-11-24T06:27:43.881850+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881850+0000","last_undegraded":"2023-11-24T06:27:43.881850+0000","last_fullsized":"2023-11-24T06:27:43.881850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893344+0000","last_change":"2023-11-24T06:27:43.888862+0000","last_active":"2023-11-24T06:27:43.893344+0000","last_peered":"2023-11-24T06:27:43.893344+0000","last_clean":"2023-11-24T06:27:43.893344+0000","last_became_active":"2023-11-24T06:27:43.888560+0000","last_became_peered":"2023-11-24T06:27:43.888560+0000","last_unstale":"2023-11-24T06:27:43.893344+0000","last_undegraded":"2023-11-24T06:27:43.893344+0000","last_fullsized":"2023-11-24T06:27:43.893344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873668+0000","last_change":"2023-11-24T06:27:43.873668+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873668+0000","last_undegraded":"2023-11-24T06:27:43.873668+0000","last_fullsized":"2023-11-24T06:27:43.873668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890166+0000","last_change":"2023-11-24T06:27:43.886888+0000","last_active":"2023-11-24T06:27:43.890166+0000","last_peered":"2023-11-24T06:27:43.890166+0000","last_clean":"2023-11-24T06:27:43.890166+0000","last_became_active":"2023-11-24T06:27:43.886584+0000","last_became_peered":"2023-11-24T06:27:43.886584+0000","last_unstale":"2023-11-24T06:27:43.890166+0000","last_undegraded":"2023-11-24T06:27:43.890166+0000","last_fullsized":"2023-11-24T06:27:43.890166+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:27.928901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878276+0000","last_change":"2023-11-24T06:27:43.878276+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878276+0000","last_undegraded":"2023-11-24T06:27:43.878276+0000","last_fullsized":"2023-11-24T06:27:43.878276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891262+0000","last_change":"2023-11-24T06:27:43.887129+0000","last_active":"2023-11-24T06:27:43.891262+0000","last_peered":"2023-11-24T06:27:43.891262+0000","last_clean":"2023-11-24T06:27:43.891262+0000","last_became_active":"2023-11-24T06:27:43.886873+0000","last_became_peered":"2023-11-24T06:27:43.886873+0000","last_unstale":"2023-11-24T06:27:43.891262+0000","last_undegraded":"2023-11-24T06:27:43.891262+0000","last_fullsized":"2023-11-24T06:27:43.891262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875846+0000","last_change":"2023-11-24T06:27:43.875846+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875846+0000","last_undegraded":"2023-11-24T06:27:43.875846+0000","last_fullsized":"2023-11-24T06:27:43.875846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893615+0000","last_change":"2023-11-24T06:27:43.891037+0000","last_active":"2023-11-24T06:27:43.893615+0000","last_peered":"2023-11-24T06:27:43.893615+0000","last_clean":"2023-11-24T06:27:43.893615+0000","last_became_active":"2023-11-24T06:27:43.890756+0000","last_became_peered":"2023-11-24T06:27:43.890756+0000","last_unstale":"2023-11-24T06:27:43.893615+0000","last_undegraded":"2023-11-24T06:27:43.893615+0000","last_fullsized":"2023-11-24T06:27:43.893615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:56.949114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877698+0000","last_change":"2023-11-24T06:27:43.877698+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877698+0000","last_undegraded":"2023-11-24T06:27:43.877698+0000","last_fullsized":"2023-11-24T06:27:43.877698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891118+0000","last_change":"2023-11-24T06:27:43.887228+0000","last_active":"2023-11-24T06:27:43.891118+0000","last_peered":"2023-11-24T06:27:43.891118+0000","last_clean":"2023-11-24T06:27:43.891118+0000","last_became_active":"2023-11-24T06:27:43.886820+0000","last_became_peered":"2023-11-24T06:27:43.886820+0000","last_unstale":"2023-11-24T06:27:43.891118+0000","last_undegraded":"2023-11-24T06:27:43.891118+0000","last_fullsized":"2023-11-24T06:27:43.891118+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:05.515434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875335+0000","last_change":"2023-11-24T06:27:43.875335+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875335+0000","last_undegraded":"2023-11-24T06:27:43.875335+0000","last_fullsized":"2023-11-24T06:27:43.875335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.881677+0000","last_change":"2023-11-24T06:27:43.880061+0000","last_active":"2023-11-24T06:27:43.881677+0000","last_peered":"2023-11-24T06:27:43.881677+0000","last_clean":"2023-11-24T06:27:43.881677+0000","last_became_active":"2023-11-24T06:27:43.879790+0000","last_became_peered":"2023-11-24T06:27:43.879790+0000","last_unstale":"2023-11-24T06:27:43.881677+0000","last_undegraded":"2023-11-24T06:27:43.881677+0000","last_fullsized":"2023-11-24T06:27:43.881677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:16.744160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881584+0000","last_change":"2023-11-24T06:27:43.881584+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881584+0000","last_undegraded":"2023-11-24T06:27:43.881584+0000","last_fullsized":"2023-11-24T06:27:43.881584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885337+0000","last_change":"2023-11-24T06:27:43.881814+0000","last_active":"2023-11-24T06:27:43.885337+0000","last_peered":"2023-11-24T06:27:43.885337+0000","last_clean":"2023-11-24T06:27:43.885337+0000","last_became_active":"2023-11-24T06:27:43.881477+0000","last_became_peered":"2023-11-24T06:27:43.881477+0000","last_unstale":"2023-11-24T06:27:43.885337+0000","last_undegraded":"2023-11-24T06:27:43.885337+0000","last_fullsized":"2023-11-24T06:27:43.885337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874261+0000","last_change":"2023-11-24T06:27:43.874261+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874261+0000","last_undegraded":"2023-11-24T06:27:43.874261+0000","last_fullsized":"2023-11-24T06:27:43.874261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.888453+0000","last_change":"2023-11-24T06:27:43.885895+0000","last_active":"2023-11-24T06:27:43.888453+0000","last_peered":"2023-11-24T06:27:43.888453+0000","last_clean":"2023-11-24T06:27:43.888453+0000","last_became_active":"2023-11-24T06:27:43.885623+0000","last_became_peered":"2023-11-24T06:27:43.885623+0000","last_unstale":"2023-11-24T06:27:43.888453+0000","last_undegraded":"2023-11-24T06:27:43.888453+0000","last_fullsized":"2023-11-24T06:27:43.888453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874451+0000","last_change":"2023-11-24T06:27:43.874451+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874451+0000","last_undegraded":"2023-11-24T06:27:43.874451+0000","last_fullsized":"2023-11-24T06:27:43.874451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890940+0000","last_change":"2023-11-24T06:27:43.887376+0000","last_active":"2023-11-24T06:27:43.890940+0000","last_peered":"2023-11-24T06:27:43.890940+0000","last_clean":"2023-11-24T06:27:43.890940+0000","last_became_active":"2023-11-24T06:27:43.887125+0000","last_became_peered":"2023-11-24T06:27:43.887125+0000","last_unstale":"2023-11-24T06:27:43.890940+0000","last_undegraded":"2023-11-24T06:27:43.890940+0000","last_fullsized":"2023-11-24T06:27:43.890940+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:55.387634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878320+0000","last_change":"2023-11-24T06:27:43.878320+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878320+0000","last_undegraded":"2023-11-24T06:27:43.878320+0000","last_fullsized":"2023-11-24T06:27:43.878320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890172+0000","last_change":"2023-11-24T06:27:43.887382+0000","last_active":"2023-11-24T06:27:43.890172+0000","last_peered":"2023-11-24T06:27:43.890172+0000","last_clean":"2023-11-24T06:27:43.890172+0000","last_became_active":"2023-11-24T06:27:43.887150+0000","last_became_peered":"2023-11-24T06:27:43.887150+0000","last_unstale":"2023-11-24T06:27:43.890172+0000","last_undegraded":"2023-11-24T06:27:43.890172+0000","last_fullsized":"2023-11-24T06:27:43.890172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:15.636561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876109+0000","last_change":"2023-11-24T06:27:43.876109+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876109+0000","last_undegraded":"2023-11-24T06:27:43.876109+0000","last_fullsized":"2023-11-24T06:27:43.876109+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887186+0000","last_change":"2023-11-24T06:27:43.884248+0000","last_active":"2023-11-24T06:27:43.887186+0000","last_peered":"2023-11-24T06:27:43.887186+0000","last_clean":"2023-11-24T06:27:43.887186+0000","last_became_active":"2023-11-24T06:27:43.883575+0000","last_became_peered":"2023-11-24T06:27:43.883575+0000","last_unstale":"2023-11-24T06:27:43.887186+0000","last_undegraded":"2023-11-24T06:27:43.887186+0000","last_fullsized":"2023-11-24T06:27:43.887186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:17:46.473315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874503+0000","last_change":"2023-11-24T06:27:43.874503+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874503+0000","last_undegraded":"2023-11-24T06:27:43.874503+0000","last_fullsized":"2023-11-24T06:27:43.874503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.896237+0000","last_change":"2023-11-24T06:27:43.890800+0000","last_active":"2023-11-24T06:27:43.896237+0000","last_peered":"2023-11-24T06:27:43.896237+0000","last_clean":"2023-11-24T06:27:43.896237+0000","last_became_active":"2023-11-24T06:27:43.890508+0000","last_became_peered":"2023-11-24T06:27:43.890508+0000","last_unstale":"2023-11-24T06:27:43.896237+0000","last_undegraded":"2023-11-24T06:27:43.896237+0000","last_fullsized":"2023-11-24T06:27:43.896237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877020+0000","last_change":"2023-11-24T06:27:43.877020+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877020+0000","last_undegraded":"2023-11-24T06:27:43.877020+0000","last_fullsized":"2023-11-24T06:27:43.877020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893569+0000","last_change":"2023-11-24T06:27:43.888453+0000","last_active":"2023-11-24T06:27:43.893569+0000","last_peered":"2023-11-24T06:27:43.893569+0000","last_clean":"2023-11-24T06:27:43.893569+0000","last_became_active":"2023-11-24T06:27:43.888169+0000","last_became_peered":"2023-11-24T06:27:43.888169+0000","last_unstale":"2023-11-24T06:27:43.893569+0000","last_undegraded":"2023-11-24T06:27:43.893569+0000","last_fullsized":"2023-11-24T06:27:43.893569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:43.753440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880509+0000","last_change":"2023-11-24T06:27:43.880509+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880509+0000","last_undegraded":"2023-11-24T06:27:43.880509+0000","last_fullsized":"2023-11-24T06:27:43.880509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893367+0000","last_change":"2023-11-24T06:27:43.886183+0000","last_active":"2023-11-24T06:27:43.893367+0000","last_peered":"2023-11-24T06:27:43.893367+0000","last_clean":"2023-11-24T06:27:43.893367+0000","last_became_active":"2023-11-24T06:27:43.885892+0000","last_became_peered":"2023-11-24T06:27:43.885892+0000","last_unstale":"2023-11-24T06:27:43.893367+0000","last_undegraded":"2023-11-24T06:27:43.893367+0000","last_fullsized":"2023-11-24T06:27:43.893367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:09.952050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884607+0000","last_change":"2023-11-24T06:27:43.884607+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884607+0000","last_undegraded":"2023-11-24T06:27:43.884607+0000","last_fullsized":"2023-11-24T06:27:43.884607+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893378+0000","last_change":"2023-11-24T06:27:43.890788+0000","last_active":"2023-11-24T06:27:43.893378+0000","last_peered":"2023-11-24T06:27:43.893378+0000","last_clean":"2023-11-24T06:27:43.893378+0000","last_became_active":"2023-11-24T06:27:43.890453+0000","last_became_peered":"2023-11-24T06:27:43.890453+0000","last_unstale":"2023-11-24T06:27:43.893378+0000","last_undegraded":"2023-11-24T06:27:43.893378+0000","last_fullsized":"2023-11-24T06:27:43.893378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877120+0000","last_change":"2023-11-24T06:27:43.877120+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877120+0000","last_undegraded":"2023-11-24T06:27:43.877120+0000","last_fullsized":"2023-11-24T06:27:43.877120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893473+0000","last_change":"2023-11-24T06:27:43.889734+0000","last_active":"2023-11-24T06:27:43.893473+0000","last_peered":"2023-11-24T06:27:43.893473+0000","last_clean":"2023-11-24T06:27:43.893473+0000","last_became_active":"2023-11-24T06:27:43.889470+0000","last_became_peered":"2023-11-24T06:27:43.889470+0000","last_unstale":"2023-11-24T06:27:43.893473+0000","last_undegraded":"2023-11-24T06:27:43.893473+0000","last_fullsized":"2023-11-24T06:27:43.893473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:45.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877591+0000","last_change":"2023-11-24T06:27:43.877591+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877591+0000","last_undegraded":"2023-11-24T06:27:43.877591+0000","last_fullsized":"2023-11-24T06:27:43.877591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891004+0000","last_change":"2023-11-24T06:27:43.889511+0000","last_active":"2023-11-24T06:27:43.891004+0000","last_peered":"2023-11-24T06:27:43.891004+0000","last_clean":"2023-11-24T06:27:43.891004+0000","last_became_active":"2023-11-24T06:27:43.889234+0000","last_became_peered":"2023-11-24T06:27:43.889234+0000","last_unstale":"2023-11-24T06:27:43.891004+0000","last_undegraded":"2023-11-24T06:27:43.891004+0000","last_fullsized":"2023-11-24T06:27:43.891004+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:07.967959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875653+0000","last_change":"2023-11-24T06:27:43.875653+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875653+0000","last_undegraded":"2023-11-24T06:27:43.875653+0000","last_fullsized":"2023-11-24T06:27:43.875653+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887987+0000","last_change":"2023-11-24T06:27:43.886417+0000","last_active":"2023-11-24T06:27:43.887987+0000","last_peered":"2023-11-24T06:27:43.887987+0000","last_clean":"2023-11-24T06:27:43.887987+0000","last_became_active":"2023-11-24T06:27:43.886110+0000","last_became_peered":"2023-11-24T06:27:43.886110+0000","last_unstale":"2023-11-24T06:27:43.887987+0000","last_undegraded":"2023-11-24T06:27:43.887987+0000","last_fullsized":"2023-11-24T06:27:43.887987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:35.558430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883673+0000","last_change":"2023-11-24T06:27:43.883673+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883673+0000","last_undegraded":"2023-11-24T06:27:43.883673+0000","last_fullsized":"2023-11-24T06:27:43.883673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891002+0000","last_change":"2023-11-24T06:27:43.887202+0000","last_active":"2023-11-24T06:27:43.891002+0000","last_peered":"2023-11-24T06:27:43.891002+0000","last_clean":"2023-11-24T06:27:43.891002+0000","last_became_active":"2023-11-24T06:27:43.886926+0000","last_became_peered":"2023-11-24T06:27:43.886926+0000","last_unstale":"2023-11-24T06:27:43.891002+0000","last_undegraded":"2023-11-24T06:27:43.891002+0000","last_fullsized":"2023-11-24T06:27:43.891002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877257+0000","last_change":"2023-11-24T06:27:43.877257+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877257+0000","last_undegraded":"2023-11-24T06:27:43.877257+0000","last_fullsized":"2023-11-24T06:27:43.877257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893321+0000","last_change":"2023-11-24T06:27:43.885639+0000","last_active":"2023-11-24T06:27:43.893321+0000","last_peered":"2023-11-24T06:27:43.893321+0000","last_clean":"2023-11-24T06:27:43.893321+0000","last_became_active":"2023-11-24T06:27:43.885321+0000","last_became_peered":"2023-11-24T06:27:43.885321+0000","last_unstale":"2023-11-24T06:27:43.893321+0000","last_undegraded":"2023-11-24T06:27:43.893321+0000","last_fullsized":"2023-11-24T06:27:43.893321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874894+0000","last_change":"2023-11-24T06:27:43.874894+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874894+0000","last_undegraded":"2023-11-24T06:27:43.874894+0000","last_fullsized":"2023-11-24T06:27:43.874894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893401+0000","last_change":"2023-11-24T06:27:43.888919+0000","last_active":"2023-11-24T06:27:43.893401+0000","last_peered":"2023-11-24T06:27:43.893401+0000","last_clean":"2023-11-24T06:27:43.893401+0000","last_became_active":"2023-11-24T06:27:43.888593+0000","last_became_peered":"2023-11-24T06:27:43.888593+0000","last_unstale":"2023-11-24T06:27:43.893401+0000","last_undegraded":"2023-11-24T06:27:43.893401+0000","last_fullsized":"2023-11-24T06:27:43.893401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:01.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875691+0000","last_change":"2023-11-24T06:27:43.875691+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875691+0000","last_undegraded":"2023-11-24T06:27:43.875691+0000","last_fullsized":"2023-11-24T06:27:43.875691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893416+0000","last_change":"2023-11-24T06:27:43.889027+0000","last_active":"2023-11-24T06:27:43.893416+0000","last_peered":"2023-11-24T06:27:43.893416+0000","last_clean":"2023-11-24T06:27:43.893416+0000","last_became_active":"2023-11-24T06:27:43.888756+0000","last_became_peered":"2023-11-24T06:27:43.888756+0000","last_unstale":"2023-11-24T06:27:43.893416+0000","last_undegraded":"2023-11-24T06:27:43.893416+0000","last_fullsized":"2023-11-24T06:27:43.893416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:59.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877954+0000","last_change":"2023-11-24T06:27:43.877954+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877954+0000","last_undegraded":"2023-11-24T06:27:43.877954+0000","last_fullsized":"2023-11-24T06:27:43.877954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.896102+0000","last_change":"2023-11-24T06:27:43.889915+0000","last_active":"2023-11-24T06:27:43.896102+0000","last_peered":"2023-11-24T06:27:43.896102+0000","last_clean":"2023-11-24T06:27:43.896102+0000","last_became_active":"2023-11-24T06:27:43.889576+0000","last_became_peered":"2023-11-24T06:27:43.889576+0000","last_unstale":"2023-11-24T06:27:43.896102+0000","last_undegraded":"2023-11-24T06:27:43.896102+0000","last_fullsized":"2023-11-24T06:27:43.896102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:09.418277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878445+0000","last_change":"2023-11-24T06:27:43.878445+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878445+0000","last_undegraded":"2023-11-24T06:27:43.878445+0000","last_fullsized":"2023-11-24T06:27:43.878445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885063+0000","last_change":"2023-11-24T06:27:43.882518+0000","last_active":"2023-11-24T06:27:43.885063+0000","last_peered":"2023-11-24T06:27:43.885063+0000","last_clean":"2023-11-24T06:27:43.885063+0000","last_became_active":"2023-11-24T06:27:43.882224+0000","last_became_peered":"2023-11-24T06:27:43.882224+0000","last_unstale":"2023-11-24T06:27:43.885063+0000","last_undegraded":"2023-11-24T06:27:43.885063+0000","last_fullsized":"2023-11-24T06:27:43.885063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.872822+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:27:42.872822+0000","last_peered":"2023-11-24T06:27:42.872822+0000","last_clean":"2023-11-24T06:27:42.872822+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:27:42.872822+0000","last_undegraded":"2023-11-24T06:27:42.872822+0000","last_fullsized":"2023-11-24T06:27:42.872822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.77600000000000002,"15min":0.86599999999999999},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.87,"5min":1.373,"15min":1.373},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.411},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.345,"15min":1.345},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.80700000000000005,"5min":1.286,"15min":1.286},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46800000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":1.379,"15min":1.379},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.376},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.316}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.315,"15min":0.313},"min":{"1min":0.20899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.313},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38900000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51200000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.374}]},{"osd":4,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":1.028,"15min":1.028},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91600000000000004,"5min":1.01,"15min":1.01},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.0369999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":1,"15min":1},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.433}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.751},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.496}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.377},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:49.289 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:49.443 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.440+0000 7f09d1263700 1 Processor -- start 2023-11-24T06:27:49.444 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.440+0000 7f09d1263700 1 -- start start 2023-11-24T06:27:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09d1263700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09cc0b49b0 0x7f09cc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09d1263700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09d1263700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0cc5c0 con 0x7f09cc151500 2023-11-24T06:27:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09d1263700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc071d70 con 0x7f09cc0b49b0 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09d1263700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc15e820 con 0x7f09cc15de50 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09ca7fc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09cc0b49b0 0x7f09cc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09cb7fe700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36730/0 (socket says 172.21.15.114:36730) 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.441+0000 7f09cb7fe700 1 -- 172.21.15.114:0/130556557 learned_addr learned my addr 172.21.15.114:0/130556557 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09ca7fc700 1 -- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 msgr2=0x7f09cc15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09ca7fc700 1 --2- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09ca7fc700 1 -- 172.21.15.114:0/130556557 >> v1:172.21.15.172:6789/0 conn(0x7f09cc151500 legacy=0x7f09cc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09ca7fc700 1 -- 172.21.15.114:0/130556557 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0717b0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09ca7fc700 1 --2- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009840 tx=0x7f09c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e18e11cae8bad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09c9ffb700 1 -- 172.21.15.114:0/130556557 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0011830 con 0x7f09cc0b49b0 2023-11-24T06:27:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09c9ffb700 1 -- 172.21.15.114:0/130556557 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0011990 con 0x7f09cc0b49b0 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09c9ffb700 1 -- 172.21.15.114:0/130556557 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0011d20 con 0x7f09cc0b49b0 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09d1263700 1 -- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc0b49b0 msgr2=0x7f09cc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.442+0000 7f09d1263700 1 --2- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009840 tx=0x7f09c0009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 -- 172.21.15.114:0/130556557 shutdown_connections 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 --2- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 --2- 172.21.15.114:0/130556557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 -- 172.21.15.114:0/130556557 >> 172.21.15.114:0/130556557 conn(0x7f09cc0b1740 msgr2=0x7f09cc0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 -- 172.21.15.114:0/130556557 shutdown_connections 2023-11-24T06:27:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.443+0000 7f09d1263700 1 -- 172.21.15.114:0/130556557 wait complete. 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.444+0000 7f09d1263700 1 Processor -- start 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.444+0000 7f09d1263700 1 -- start start 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc151500 0x7f09cc1f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1532f0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc06dea0 con 0x7f09cc151500 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc152390 con 0x7f09cc15de50 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42188/0 (socket says 172.21.15.114:42188) 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 -- 172.21.15.114:0/1681538305 learned_addr learned my addr 172.21.15.114:0/1681538305 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09ca7fc700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc151500 0x7f09cc1f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09cb7fe700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 -- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 msgr2=0x7f09cc15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 -- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc151500 msgr2=0x7f09cc1f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc151500 0x7f09cc1f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 -- 172.21.15.114:0/1681538305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00094a0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09caffd700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c4e0 tx=0x7f09bc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0052d0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc005430 con 0x7f09cc0b49b0 2023-11-24T06:27:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc152670 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0038c0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.445+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc152c30 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.446+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc153610 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.447+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f09bc003a20 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.447+0000 7f09b37fe700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 0x7f09b806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.447+0000 7f09ca7fc700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 0x7f09b806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.447+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09bc088c80 con 0x7f09cc0b49b0 2023-11-24T06:27:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.448+0000 7f09ca7fc700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 0x7f09b806d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.452+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09bc053ab0 con 0x7f09cc0b49b0 2023-11-24T06:27:49.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.635+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09cc0714a0 con 0x7f09b806b080 2023-11-24T06:27:49.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.643+0000 7f09b37fe700 1 -- 172.21.15.114:0/1681538305 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283983 (secure 0 0 0) 0x7f09c0011c00 con 0x7f09b806b080 2023-11-24T06:27:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 msgr2=0x7f09b806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 0x7f09b806d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0008030 comp rx=0 tx=0).stop 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 msgr2=0x7f09cc06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c4e0 tx=0x7f09bc00c7f0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 shutdown_connections 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f09b806b080 0x7f09b806d540 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09cc15de50 0x7f09cc15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09cc0b49b0 0x7f09cc06d960 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 --2- 172.21.15.114:0/1681538305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09cc151500 0x7f09cc1f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 >> 172.21.15.114:0/1681538305 conn(0x7f09cc0b1740 msgr2=0x7f09cc0b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 shutdown_connections 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.646+0000 7f09d1263700 1 -- 172.21.15.114:0/1681538305 wait complete. 2023-11-24T06:27:49.659 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:27:49.662 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-24T06:27:47.752814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222584,"kb_used_data":6396,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752136,"statfs":{"total":773094113280,"available":772866187264,"internally_reserved":0,"allocated":6549504,"data_stored":3388245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.881873"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877928+0000","last_change":"2023-11-24T06:27:43.877928+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877928+0000","last_undegraded":"2023-11-24T06:27:43.877928+0000","last_fullsized":"2023-11-24T06:27:43.877928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878401+0000","last_change":"2023-11-24T06:27:43.878401+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878401+0000","last_undegraded":"2023-11-24T06:27:43.878401+0000","last_fullsized":"2023-11-24T06:27:43.878401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876797+0000","last_change":"2023-11-24T06:27:43.876797+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876797+0000","last_undegraded":"2023-11-24T06:27:43.876797+0000","last_fullsized":"2023-11-24T06:27:43.876797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880382+0000","last_change":"2023-11-24T06:27:43.880382+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880382+0000","last_undegraded":"2023-11-24T06:27:43.880382+0000","last_fullsized":"2023-11-24T06:27:43.880382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875033+0000","last_change":"2023-11-24T06:27:43.875033+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875033+0000","last_undegraded":"2023-11-24T06:27:43.875033+0000","last_fullsized":"2023-11-24T06:27:43.875033+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881134+0000","last_change":"2023-11-24T06:27:43.881134+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881134+0000","last_undegraded":"2023-11-24T06:27:43.881134+0000","last_fullsized":"2023-11-24T06:27:43.881134+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884229+0000","last_change":"2023-11-24T06:27:43.884229+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884229+0000","last_undegraded":"2023-11-24T06:27:43.884229+0000","last_fullsized":"2023-11-24T06:27:43.884229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878821+0000","last_change":"2023-11-24T06:27:43.878821+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878821+0000","last_undegraded":"2023-11-24T06:27:43.878821+0000","last_fullsized":"2023-11-24T06:27:43.878821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881743+0000","last_change":"2023-11-24T06:27:43.881743+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881743+0000","last_undegraded":"2023-11-24T06:27:43.881743+0000","last_fullsized":"2023-11-24T06:27:43.881743+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875949+0000","last_change":"2023-11-24T06:27:43.875949+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875949+0000","last_undegraded":"2023-11-24T06:27:43.875949+0000","last_fullsized":"2023-11-24T06:27:43.875949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876187+0000","last_change":"2023-11-24T06:27:43.876187+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876187+0000","last_undegraded":"2023-11-24T06:27:43.876187+0000","last_fullsized":"2023-11-24T06:27:43.876187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878485+0000","last_change":"2023-11-24T06:27:43.878485+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878485+0000","last_undegraded":"2023-11-24T06:27:43.878485+0000","last_fullsized":"2023-11-24T06:27:43.878485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874177+0000","last_change":"2023-11-24T06:27:43.874177+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874177+0000","last_undegraded":"2023-11-24T06:27:43.874177+0000","last_fullsized":"2023-11-24T06:27:43.874177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882087+0000","last_change":"2023-11-24T06:27:43.882087+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882087+0000","last_undegraded":"2023-11-24T06:27:43.882087+0000","last_fullsized":"2023-11-24T06:27:43.882087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883840+0000","last_change":"2023-11-24T06:27:43.883840+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883840+0000","last_undegraded":"2023-11-24T06:27:43.883840+0000","last_fullsized":"2023-11-24T06:27:43.883840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880935+0000","last_change":"2023-11-24T06:27:43.880935+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880935+0000","last_undegraded":"2023-11-24T06:27:43.880935+0000","last_fullsized":"2023-11-24T06:27:43.880935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877635+0000","last_change":"2023-11-24T06:27:43.877635+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877635+0000","last_undegraded":"2023-11-24T06:27:43.877635+0000","last_fullsized":"2023-11-24T06:27:43.877635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875034+0000","last_change":"2023-11-24T06:27:43.875034+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875034+0000","last_undegraded":"2023-11-24T06:27:43.875034+0000","last_fullsized":"2023-11-24T06:27:43.875034+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875072+0000","last_change":"2023-11-24T06:27:43.875072+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875072+0000","last_undegraded":"2023-11-24T06:27:43.875072+0000","last_fullsized":"2023-11-24T06:27:43.875072+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876540+0000","last_change":"2023-11-24T06:27:43.876540+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876540+0000","last_undegraded":"2023-11-24T06:27:43.876540+0000","last_fullsized":"2023-11-24T06:27:43.876540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875751+0000","last_change":"2023-11-24T06:27:43.875751+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875751+0000","last_undegraded":"2023-11-24T06:27:43.875751+0000","last_fullsized":"2023-11-24T06:27:43.875751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878158+0000","last_change":"2023-11-24T06:27:43.878158+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878158+0000","last_undegraded":"2023-11-24T06:27:43.878158+0000","last_fullsized":"2023-11-24T06:27:43.878158+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873638+0000","last_change":"2023-11-24T06:27:43.873638+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873638+0000","last_undegraded":"2023-11-24T06:27:43.873638+0000","last_fullsized":"2023-11-24T06:27:43.873638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875332+0000","last_change":"2023-11-24T06:27:43.875332+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875332+0000","last_undegraded":"2023-11-24T06:27:43.875332+0000","last_fullsized":"2023-11-24T06:27:43.875332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874699+0000","last_change":"2023-11-24T06:27:43.874699+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874699+0000","last_undegraded":"2023-11-24T06:27:43.874699+0000","last_fullsized":"2023-11-24T06:27:43.874699+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878658+0000","last_change":"2023-11-24T06:27:43.878658+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878658+0000","last_undegraded":"2023-11-24T06:27:43.878658+0000","last_fullsized":"2023-11-24T06:27:43.878658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884489+0000","last_change":"2023-11-24T06:27:43.884489+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884489+0000","last_undegraded":"2023-11-24T06:27:43.884489+0000","last_fullsized":"2023-11-24T06:27:43.884489+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874694+0000","last_change":"2023-11-24T06:27:43.874694+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874694+0000","last_undegraded":"2023-11-24T06:27:43.874694+0000","last_fullsized":"2023-11-24T06:27:43.874694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877093+0000","last_change":"2023-11-24T06:27:43.877093+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877093+0000","last_undegraded":"2023-11-24T06:27:43.877093+0000","last_fullsized":"2023-11-24T06:27:43.877093+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874041+0000","last_change":"2023-11-24T06:27:43.874041+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874041+0000","last_undegraded":"2023-11-24T06:27:43.874041+0000","last_fullsized":"2023-11-24T06:27:43.874041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875031+0000","last_change":"2023-11-24T06:27:43.875031+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875031+0000","last_undegraded":"2023-11-24T06:27:43.875031+0000","last_fullsized":"2023-11-24T06:27:43.875031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880505+0000","last_change":"2023-11-24T06:27:43.880505+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880505+0000","last_undegraded":"2023-11-24T06:27:43.880505+0000","last_fullsized":"2023-11-24T06:27:43.880505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882036+0000","last_change":"2023-11-24T06:27:43.882036+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882036+0000","last_undegraded":"2023-11-24T06:27:43.882036+0000","last_fullsized":"2023-11-24T06:27:43.882036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890188+0000","last_change":"2023-11-24T06:27:43.887983+0000","last_active":"2023-11-24T06:27:43.890188+0000","last_peered":"2023-11-24T06:27:43.890188+0000","last_clean":"2023-11-24T06:27:43.890188+0000","last_became_active":"2023-11-24T06:27:43.887687+0000","last_became_peered":"2023-11-24T06:27:43.887687+0000","last_unstale":"2023-11-24T06:27:43.890188+0000","last_undegraded":"2023-11-24T06:27:43.890188+0000","last_fullsized":"2023-11-24T06:27:43.890188+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:23.658351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876827+0000","last_change":"2023-11-24T06:27:43.876827+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876827+0000","last_undegraded":"2023-11-24T06:27:43.876827+0000","last_fullsized":"2023-11-24T06:27:43.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893472+0000","last_change":"2023-11-24T06:27:43.887478+0000","last_active":"2023-11-24T06:27:43.893472+0000","last_peered":"2023-11-24T06:27:43.893472+0000","last_clean":"2023-11-24T06:27:43.893472+0000","last_became_active":"2023-11-24T06:27:43.886904+0000","last_became_peered":"2023-11-24T06:27:43.886904+0000","last_unstale":"2023-11-24T06:27:43.893472+0000","last_undegraded":"2023-11-24T06:27:43.893472+0000","last_fullsized":"2023-11-24T06:27:43.893472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:17.375576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882394+0000","last_change":"2023-11-24T06:27:43.882394+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882394+0000","last_undegraded":"2023-11-24T06:27:43.882394+0000","last_fullsized":"2023-11-24T06:27:43.882394+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.899278+0000","last_change":"2023-11-24T06:27:43.893948+0000","last_active":"2023-11-24T06:27:43.899278+0000","last_peered":"2023-11-24T06:27:43.899278+0000","last_clean":"2023-11-24T06:27:43.899278+0000","last_became_active":"2023-11-24T06:27:43.893661+0000","last_became_peered":"2023-11-24T06:27:43.893661+0000","last_unstale":"2023-11-24T06:27:43.899278+0000","last_undegraded":"2023-11-24T06:27:43.899278+0000","last_fullsized":"2023-11-24T06:27:43.899278+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884086+0000","last_change":"2023-11-24T06:27:43.884086+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884086+0000","last_undegraded":"2023-11-24T06:27:43.884086+0000","last_fullsized":"2023-11-24T06:27:43.884086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887326+0000","last_change":"2023-11-24T06:27:43.884164+0000","last_active":"2023-11-24T06:27:43.887326+0000","last_peered":"2023-11-24T06:27:43.887326+0000","last_clean":"2023-11-24T06:27:43.887326+0000","last_became_active":"2023-11-24T06:27:43.883886+0000","last_became_peered":"2023-11-24T06:27:43.883886+0000","last_unstale":"2023-11-24T06:27:43.887326+0000","last_undegraded":"2023-11-24T06:27:43.887326+0000","last_fullsized":"2023-11-24T06:27:43.887326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:35.939429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877665+0000","last_change":"2023-11-24T06:27:43.877665+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877665+0000","last_undegraded":"2023-11-24T06:27:43.877665+0000","last_fullsized":"2023-11-24T06:27:43.877665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891238+0000","last_change":"2023-11-24T06:27:43.887860+0000","last_active":"2023-11-24T06:27:43.891238+0000","last_peered":"2023-11-24T06:27:43.891238+0000","last_clean":"2023-11-24T06:27:43.891238+0000","last_became_active":"2023-11-24T06:27:43.887522+0000","last_became_peered":"2023-11-24T06:27:43.887522+0000","last_unstale":"2023-11-24T06:27:43.891238+0000","last_undegraded":"2023-11-24T06:27:43.891238+0000","last_fullsized":"2023-11-24T06:27:43.891238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:18.289207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876826+0000","last_change":"2023-11-24T06:27:43.876826+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876826+0000","last_undegraded":"2023-11-24T06:27:43.876826+0000","last_fullsized":"2023-11-24T06:27:43.876826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890657+0000","last_change":"2023-11-24T06:27:43.888671+0000","last_active":"2023-11-24T06:27:43.890657+0000","last_peered":"2023-11-24T06:27:43.890657+0000","last_clean":"2023-11-24T06:27:43.890657+0000","last_became_active":"2023-11-24T06:27:43.888398+0000","last_became_peered":"2023-11-24T06:27:43.888398+0000","last_unstale":"2023-11-24T06:27:43.890657+0000","last_undegraded":"2023-11-24T06:27:43.890657+0000","last_fullsized":"2023-11-24T06:27:43.890657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:40.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873693+0000","last_change":"2023-11-24T06:27:43.873693+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873693+0000","last_undegraded":"2023-11-24T06:27:43.873693+0000","last_fullsized":"2023-11-24T06:27:43.873693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893351+0000","last_change":"2023-11-24T06:27:43.888597+0000","last_active":"2023-11-24T06:27:43.893351+0000","last_peered":"2023-11-24T06:27:43.893351+0000","last_clean":"2023-11-24T06:27:43.893351+0000","last_became_active":"2023-11-24T06:27:43.888055+0000","last_became_peered":"2023-11-24T06:27:43.888055+0000","last_unstale":"2023-11-24T06:27:43.893351+0000","last_undegraded":"2023-11-24T06:27:43.893351+0000","last_fullsized":"2023-11-24T06:27:43.893351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878321+0000","last_change":"2023-11-24T06:27:43.878321+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878321+0000","last_undegraded":"2023-11-24T06:27:43.878321+0000","last_fullsized":"2023-11-24T06:27:43.878321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893456+0000","last_change":"2023-11-24T06:27:43.890543+0000","last_active":"2023-11-24T06:27:43.893456+0000","last_peered":"2023-11-24T06:27:43.893456+0000","last_clean":"2023-11-24T06:27:43.893456+0000","last_became_active":"2023-11-24T06:27:43.890213+0000","last_became_peered":"2023-11-24T06:27:43.890213+0000","last_unstale":"2023-11-24T06:27:43.893456+0000","last_undegraded":"2023-11-24T06:27:43.893456+0000","last_fullsized":"2023-11-24T06:27:43.893456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:21.791667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881850+0000","last_change":"2023-11-24T06:27:43.881850+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881850+0000","last_undegraded":"2023-11-24T06:27:43.881850+0000","last_fullsized":"2023-11-24T06:27:43.881850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893344+0000","last_change":"2023-11-24T06:27:43.888862+0000","last_active":"2023-11-24T06:27:43.893344+0000","last_peered":"2023-11-24T06:27:43.893344+0000","last_clean":"2023-11-24T06:27:43.893344+0000","last_became_active":"2023-11-24T06:27:43.888560+0000","last_became_peered":"2023-11-24T06:27:43.888560+0000","last_unstale":"2023-11-24T06:27:43.893344+0000","last_undegraded":"2023-11-24T06:27:43.893344+0000","last_fullsized":"2023-11-24T06:27:43.893344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873668+0000","last_change":"2023-11-24T06:27:43.873668+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873668+0000","last_undegraded":"2023-11-24T06:27:43.873668+0000","last_fullsized":"2023-11-24T06:27:43.873668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890166+0000","last_change":"2023-11-24T06:27:43.886888+0000","last_active":"2023-11-24T06:27:43.890166+0000","last_peered":"2023-11-24T06:27:43.890166+0000","last_clean":"2023-11-24T06:27:43.890166+0000","last_became_active":"2023-11-24T06:27:43.886584+0000","last_became_peered":"2023-11-24T06:27:43.886584+0000","last_unstale":"2023-11-24T06:27:43.890166+0000","last_undegraded":"2023-11-24T06:27:43.890166+0000","last_fullsized":"2023-11-24T06:27:43.890166+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:27.928901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878276+0000","last_change":"2023-11-24T06:27:43.878276+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878276+0000","last_undegraded":"2023-11-24T06:27:43.878276+0000","last_fullsized":"2023-11-24T06:27:43.878276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891262+0000","last_change":"2023-11-24T06:27:43.887129+0000","last_active":"2023-11-24T06:27:43.891262+0000","last_peered":"2023-11-24T06:27:43.891262+0000","last_clean":"2023-11-24T06:27:43.891262+0000","last_became_active":"2023-11-24T06:27:43.886873+0000","last_became_peered":"2023-11-24T06:27:43.886873+0000","last_unstale":"2023-11-24T06:27:43.891262+0000","last_undegraded":"2023-11-24T06:27:43.891262+0000","last_fullsized":"2023-11-24T06:27:43.891262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875846+0000","last_change":"2023-11-24T06:27:43.875846+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875846+0000","last_undegraded":"2023-11-24T06:27:43.875846+0000","last_fullsized":"2023-11-24T06:27:43.875846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893615+0000","last_change":"2023-11-24T06:27:43.891037+0000","last_active":"2023-11-24T06:27:43.893615+0000","last_peered":"2023-11-24T06:27:43.893615+0000","last_clean":"2023-11-24T06:27:43.893615+0000","last_became_active":"2023-11-24T06:27:43.890756+0000","last_became_peered":"2023-11-24T06:27:43.890756+0000","last_unstale":"2023-11-24T06:27:43.893615+0000","last_undegraded":"2023-11-24T06:27:43.893615+0000","last_fullsized":"2023-11-24T06:27:43.893615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:56.949114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877698+0000","last_change":"2023-11-24T06:27:43.877698+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877698+0000","last_undegraded":"2023-11-24T06:27:43.877698+0000","last_fullsized":"2023-11-24T06:27:43.877698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891118+0000","last_change":"2023-11-24T06:27:43.887228+0000","last_active":"2023-11-24T06:27:43.891118+0000","last_peered":"2023-11-24T06:27:43.891118+0000","last_clean":"2023-11-24T06:27:43.891118+0000","last_became_active":"2023-11-24T06:27:43.886820+0000","last_became_peered":"2023-11-24T06:27:43.886820+0000","last_unstale":"2023-11-24T06:27:43.891118+0000","last_undegraded":"2023-11-24T06:27:43.891118+0000","last_fullsized":"2023-11-24T06:27:43.891118+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:05.515434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875335+0000","last_change":"2023-11-24T06:27:43.875335+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875335+0000","last_undegraded":"2023-11-24T06:27:43.875335+0000","last_fullsized":"2023-11-24T06:27:43.875335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.881677+0000","last_change":"2023-11-24T06:27:43.880061+0000","last_active":"2023-11-24T06:27:43.881677+0000","last_peered":"2023-11-24T06:27:43.881677+0000","last_clean":"2023-11-24T06:27:43.881677+0000","last_became_active":"2023-11-24T06:27:43.879790+0000","last_became_peered":"2023-11-24T06:27:43.879790+0000","last_unstale":"2023-11-24T06:27:43.881677+0000","last_undegraded":"2023-11-24T06:27:43.881677+0000","last_fullsized":"2023-11-24T06:27:43.881677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:16.744160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881584+0000","last_change":"2023-11-24T06:27:43.881584+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881584+0000","last_undegraded":"2023-11-24T06:27:43.881584+0000","last_fullsized":"2023-11-24T06:27:43.881584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885337+0000","last_change":"2023-11-24T06:27:43.881814+0000","last_active":"2023-11-24T06:27:43.885337+0000","last_peered":"2023-11-24T06:27:43.885337+0000","last_clean":"2023-11-24T06:27:43.885337+0000","last_became_active":"2023-11-24T06:27:43.881477+0000","last_became_peered":"2023-11-24T06:27:43.881477+0000","last_unstale":"2023-11-24T06:27:43.885337+0000","last_undegraded":"2023-11-24T06:27:43.885337+0000","last_fullsized":"2023-11-24T06:27:43.885337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874261+0000","last_change":"2023-11-24T06:27:43.874261+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874261+0000","last_undegraded":"2023-11-24T06:27:43.874261+0000","last_fullsized":"2023-11-24T06:27:43.874261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.888453+0000","last_change":"2023-11-24T06:27:43.885895+0000","last_active":"2023-11-24T06:27:43.888453+0000","last_peered":"2023-11-24T06:27:43.888453+0000","last_clean":"2023-11-24T06:27:43.888453+0000","last_became_active":"2023-11-24T06:27:43.885623+0000","last_became_peered":"2023-11-24T06:27:43.885623+0000","last_unstale":"2023-11-24T06:27:43.888453+0000","last_undegraded":"2023-11-24T06:27:43.888453+0000","last_fullsized":"2023-11-24T06:27:43.888453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874451+0000","last_change":"2023-11-24T06:27:43.874451+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874451+0000","last_undegraded":"2023-11-24T06:27:43.874451+0000","last_fullsized":"2023-11-24T06:27:43.874451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890940+0000","last_change":"2023-11-24T06:27:43.887376+0000","last_active":"2023-11-24T06:27:43.890940+0000","last_peered":"2023-11-24T06:27:43.890940+0000","last_clean":"2023-11-24T06:27:43.890940+0000","last_became_active":"2023-11-24T06:27:43.887125+0000","last_became_peered":"2023-11-24T06:27:43.887125+0000","last_unstale":"2023-11-24T06:27:43.890940+0000","last_undegraded":"2023-11-24T06:27:43.890940+0000","last_fullsized":"2023-11-24T06:27:43.890940+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:55.387634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878320+0000","last_change":"2023-11-24T06:27:43.878320+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878320+0000","last_undegraded":"2023-11-24T06:27:43.878320+0000","last_fullsized":"2023-11-24T06:27:43.878320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.890172+0000","last_change":"2023-11-24T06:27:43.887382+0000","last_active":"2023-11-24T06:27:43.890172+0000","last_peered":"2023-11-24T06:27:43.890172+0000","last_clean":"2023-11-24T06:27:43.890172+0000","last_became_active":"2023-11-24T06:27:43.887150+0000","last_became_peered":"2023-11-24T06:27:43.887150+0000","last_unstale":"2023-11-24T06:27:43.890172+0000","last_undegraded":"2023-11-24T06:27:43.890172+0000","last_fullsized":"2023-11-24T06:27:43.890172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:15.636561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876109+0000","last_change":"2023-11-24T06:27:43.876109+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876109+0000","last_undegraded":"2023-11-24T06:27:43.876109+0000","last_fullsized":"2023-11-24T06:27:43.876109+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887186+0000","last_change":"2023-11-24T06:27:43.884248+0000","last_active":"2023-11-24T06:27:43.887186+0000","last_peered":"2023-11-24T06:27:43.887186+0000","last_clean":"2023-11-24T06:27:43.887186+0000","last_became_active":"2023-11-24T06:27:43.883575+0000","last_became_peered":"2023-11-24T06:27:43.883575+0000","last_unstale":"2023-11-24T06:27:43.887186+0000","last_undegraded":"2023-11-24T06:27:43.887186+0000","last_fullsized":"2023-11-24T06:27:43.887186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:17:46.473315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874503+0000","last_change":"2023-11-24T06:27:43.874503+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874503+0000","last_undegraded":"2023-11-24T06:27:43.874503+0000","last_fullsized":"2023-11-24T06:27:43.874503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.896237+0000","last_change":"2023-11-24T06:27:43.890800+0000","last_active":"2023-11-24T06:27:43.896237+0000","last_peered":"2023-11-24T06:27:43.896237+0000","last_clean":"2023-11-24T06:27:43.896237+0000","last_became_active":"2023-11-24T06:27:43.890508+0000","last_became_peered":"2023-11-24T06:27:43.890508+0000","last_unstale":"2023-11-24T06:27:43.896237+0000","last_undegraded":"2023-11-24T06:27:43.896237+0000","last_fullsized":"2023-11-24T06:27:43.896237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877020+0000","last_change":"2023-11-24T06:27:43.877020+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877020+0000","last_undegraded":"2023-11-24T06:27:43.877020+0000","last_fullsized":"2023-11-24T06:27:43.877020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893569+0000","last_change":"2023-11-24T06:27:43.888453+0000","last_active":"2023-11-24T06:27:43.893569+0000","last_peered":"2023-11-24T06:27:43.893569+0000","last_clean":"2023-11-24T06:27:43.893569+0000","last_became_active":"2023-11-24T06:27:43.888169+0000","last_became_peered":"2023-11-24T06:27:43.888169+0000","last_unstale":"2023-11-24T06:27:43.893569+0000","last_undegraded":"2023-11-24T06:27:43.893569+0000","last_fullsized":"2023-11-24T06:27:43.893569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:43.753440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880509+0000","last_change":"2023-11-24T06:27:43.880509+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880509+0000","last_undegraded":"2023-11-24T06:27:43.880509+0000","last_fullsized":"2023-11-24T06:27:43.880509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893367+0000","last_change":"2023-11-24T06:27:43.886183+0000","last_active":"2023-11-24T06:27:43.893367+0000","last_peered":"2023-11-24T06:27:43.893367+0000","last_clean":"2023-11-24T06:27:43.893367+0000","last_became_active":"2023-11-24T06:27:43.885892+0000","last_became_peered":"2023-11-24T06:27:43.885892+0000","last_unstale":"2023-11-24T06:27:43.893367+0000","last_undegraded":"2023-11-24T06:27:43.893367+0000","last_fullsized":"2023-11-24T06:27:43.893367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:09.952050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884607+0000","last_change":"2023-11-24T06:27:43.884607+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884607+0000","last_undegraded":"2023-11-24T06:27:43.884607+0000","last_fullsized":"2023-11-24T06:27:43.884607+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893378+0000","last_change":"2023-11-24T06:27:43.890788+0000","last_active":"2023-11-24T06:27:43.893378+0000","last_peered":"2023-11-24T06:27:43.893378+0000","last_clean":"2023-11-24T06:27:43.893378+0000","last_became_active":"2023-11-24T06:27:43.890453+0000","last_became_peered":"2023-11-24T06:27:43.890453+0000","last_unstale":"2023-11-24T06:27:43.893378+0000","last_undegraded":"2023-11-24T06:27:43.893378+0000","last_fullsized":"2023-11-24T06:27:43.893378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877120+0000","last_change":"2023-11-24T06:27:43.877120+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877120+0000","last_undegraded":"2023-11-24T06:27:43.877120+0000","last_fullsized":"2023-11-24T06:27:43.877120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893473+0000","last_change":"2023-11-24T06:27:43.889734+0000","last_active":"2023-11-24T06:27:43.893473+0000","last_peered":"2023-11-24T06:27:43.893473+0000","last_clean":"2023-11-24T06:27:43.893473+0000","last_became_active":"2023-11-24T06:27:43.889470+0000","last_became_peered":"2023-11-24T06:27:43.889470+0000","last_unstale":"2023-11-24T06:27:43.893473+0000","last_undegraded":"2023-11-24T06:27:43.893473+0000","last_fullsized":"2023-11-24T06:27:43.893473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:45.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877591+0000","last_change":"2023-11-24T06:27:43.877591+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877591+0000","last_undegraded":"2023-11-24T06:27:43.877591+0000","last_fullsized":"2023-11-24T06:27:43.877591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891004+0000","last_change":"2023-11-24T06:27:43.889511+0000","last_active":"2023-11-24T06:27:43.891004+0000","last_peered":"2023-11-24T06:27:43.891004+0000","last_clean":"2023-11-24T06:27:43.891004+0000","last_became_active":"2023-11-24T06:27:43.889234+0000","last_became_peered":"2023-11-24T06:27:43.889234+0000","last_unstale":"2023-11-24T06:27:43.891004+0000","last_undegraded":"2023-11-24T06:27:43.891004+0000","last_fullsized":"2023-11-24T06:27:43.891004+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:07.967959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875653+0000","last_change":"2023-11-24T06:27:43.875653+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875653+0000","last_undegraded":"2023-11-24T06:27:43.875653+0000","last_fullsized":"2023-11-24T06:27:43.875653+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887987+0000","last_change":"2023-11-24T06:27:43.886417+0000","last_active":"2023-11-24T06:27:43.887987+0000","last_peered":"2023-11-24T06:27:43.887987+0000","last_clean":"2023-11-24T06:27:43.887987+0000","last_became_active":"2023-11-24T06:27:43.886110+0000","last_became_peered":"2023-11-24T06:27:43.886110+0000","last_unstale":"2023-11-24T06:27:43.887987+0000","last_undegraded":"2023-11-24T06:27:43.887987+0000","last_fullsized":"2023-11-24T06:27:43.887987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:35.558430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883673+0000","last_change":"2023-11-24T06:27:43.883673+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883673+0000","last_undegraded":"2023-11-24T06:27:43.883673+0000","last_fullsized":"2023-11-24T06:27:43.883673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891002+0000","last_change":"2023-11-24T06:27:43.887202+0000","last_active":"2023-11-24T06:27:43.891002+0000","last_peered":"2023-11-24T06:27:43.891002+0000","last_clean":"2023-11-24T06:27:43.891002+0000","last_became_active":"2023-11-24T06:27:43.886926+0000","last_became_peered":"2023-11-24T06:27:43.886926+0000","last_unstale":"2023-11-24T06:27:43.891002+0000","last_undegraded":"2023-11-24T06:27:43.891002+0000","last_fullsized":"2023-11-24T06:27:43.891002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877257+0000","last_change":"2023-11-24T06:27:43.877257+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877257+0000","last_undegraded":"2023-11-24T06:27:43.877257+0000","last_fullsized":"2023-11-24T06:27:43.877257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893321+0000","last_change":"2023-11-24T06:27:43.885639+0000","last_active":"2023-11-24T06:27:43.893321+0000","last_peered":"2023-11-24T06:27:43.893321+0000","last_clean":"2023-11-24T06:27:43.893321+0000","last_became_active":"2023-11-24T06:27:43.885321+0000","last_became_peered":"2023-11-24T06:27:43.885321+0000","last_unstale":"2023-11-24T06:27:43.893321+0000","last_undegraded":"2023-11-24T06:27:43.893321+0000","last_fullsized":"2023-11-24T06:27:43.893321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874894+0000","last_change":"2023-11-24T06:27:43.874894+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874894+0000","last_undegraded":"2023-11-24T06:27:43.874894+0000","last_fullsized":"2023-11-24T06:27:43.874894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893401+0000","last_change":"2023-11-24T06:27:43.888919+0000","last_active":"2023-11-24T06:27:43.893401+0000","last_peered":"2023-11-24T06:27:43.893401+0000","last_clean":"2023-11-24T06:27:43.893401+0000","last_became_active":"2023-11-24T06:27:43.888593+0000","last_became_peered":"2023-11-24T06:27:43.888593+0000","last_unstale":"2023-11-24T06:27:43.893401+0000","last_undegraded":"2023-11-24T06:27:43.893401+0000","last_fullsized":"2023-11-24T06:27:43.893401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:01.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875691+0000","last_change":"2023-11-24T06:27:43.875691+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875691+0000","last_undegraded":"2023-11-24T06:27:43.875691+0000","last_fullsized":"2023-11-24T06:27:43.875691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893416+0000","last_change":"2023-11-24T06:27:43.889027+0000","last_active":"2023-11-24T06:27:43.893416+0000","last_peered":"2023-11-24T06:27:43.893416+0000","last_clean":"2023-11-24T06:27:43.893416+0000","last_became_active":"2023-11-24T06:27:43.888756+0000","last_became_peered":"2023-11-24T06:27:43.888756+0000","last_unstale":"2023-11-24T06:27:43.893416+0000","last_undegraded":"2023-11-24T06:27:43.893416+0000","last_fullsized":"2023-11-24T06:27:43.893416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:59.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877954+0000","last_change":"2023-11-24T06:27:43.877954+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877954+0000","last_undegraded":"2023-11-24T06:27:43.877954+0000","last_fullsized":"2023-11-24T06:27:43.877954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.896102+0000","last_change":"2023-11-24T06:27:43.889915+0000","last_active":"2023-11-24T06:27:43.896102+0000","last_peered":"2023-11-24T06:27:43.896102+0000","last_clean":"2023-11-24T06:27:43.896102+0000","last_became_active":"2023-11-24T06:27:43.889576+0000","last_became_peered":"2023-11-24T06:27:43.889576+0000","last_unstale":"2023-11-24T06:27:43.896102+0000","last_undegraded":"2023-11-24T06:27:43.896102+0000","last_fullsized":"2023-11-24T06:27:43.896102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:09.418277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878445+0000","last_change":"2023-11-24T06:27:43.878445+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878445+0000","last_undegraded":"2023-11-24T06:27:43.878445+0000","last_fullsized":"2023-11-24T06:27:43.878445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885063+0000","last_change":"2023-11-24T06:27:43.882518+0000","last_active":"2023-11-24T06:27:43.885063+0000","last_peered":"2023-11-24T06:27:43.885063+0000","last_clean":"2023-11-24T06:27:43.885063+0000","last_became_active":"2023-11-24T06:27:43.882224+0000","last_became_peered":"2023-11-24T06:27:43.882224+0000","last_unstale":"2023-11-24T06:27:43.885063+0000","last_undegraded":"2023-11-24T06:27:43.885063+0000","last_fullsized":"2023-11-24T06:27:43.885063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.872822+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:27:42.872822+0000","last_peered":"2023-11-24T06:27:42.872822+0000","last_clean":"2023-11-24T06:27:42.872822+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:27:42.872822+0000","last_undegraded":"2023-11-24T06:27:42.872822+0000","last_fullsized":"2023-11-24T06:27:42.872822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.77600000000000002,"15min":0.86599999999999999},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.87,"5min":1.373,"15min":1.373},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.411},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.345,"15min":1.345},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.80700000000000005,"5min":1.286,"15min":1.286},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46800000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":1.379,"15min":1.379},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.376},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.316}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.315,"15min":0.313},"min":{"1min":0.20899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.313},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38900000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51200000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.374}]},{"osd":4,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":1.028,"15min":1.028},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91600000000000004,"5min":1.01,"15min":1.01},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.0369999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":1,"15min":1},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.433}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.746},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.751},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.496}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.377},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:49.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:49.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.819+0000 7f670f9a6700 1 Processor -- start 2023-11-24T06:27:49.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 -- start start 2023-11-24T06:27:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f67080d2240 0x7f67080b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 0x7f67080cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080b1d40 con 0x7f670815ddd0 2023-11-24T06:27:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080b28c0 con 0x7f67080d2240 2023-11-24T06:27:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.820+0000 7f670f9a6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670806c000 con 0x7f670815e1b0 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f67080d2240 0x7f67080b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670d742700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f670815ddd0 0x7f67080d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57904/0 (socket says 172.21.15.114:57904) 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670d742700 1 -- 172.21.15.114:0/2653681532 learned_addr learned my addr 172.21.15.114:0/2653681532 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670df43700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 0x7f67080cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 -- 172.21.15.114:0/2653681532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 msgr2=0x7f67080cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 0x7f67080cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 -- 172.21.15.114:0/2653681532 >> v1:172.21.15.114:6789/0 conn(0x7f670815ddd0 legacy=0x7f67080d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f66fffff700 1 -- 172.21.15.114:0/2653681532 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632731716 0 0) 0x7f67080b1d40 con 0x7f670815ddd0 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 -- 172.21.15.114:0/2653681532 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080b1f90 con 0x7f67080d2240 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670df43700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 0x7f67080cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f670cf41700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67080d2240 0x7f67080b1680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6704009870 tx=0x7f6704009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad92d99e51f66e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f66fffff700 1 -- 172.21.15.114:0/2653681532 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011830 con 0x7f67080d2240 2023-11-24T06:27:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f66fffff700 1 -- 172.21.15.114:0/2653681532 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704011990 con 0x7f67080d2240 2023-11-24T06:27:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.821+0000 7f66fffff700 1 -- 172.21.15.114:0/2653681532 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011d20 con 0x7f67080d2240 2023-11-24T06:27:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 -- 172.21.15.114:0/2653681532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67080d2240 msgr2=0x7f67080b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67080d2240 0x7f67080b1680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6704009870 tx=0x7f6704009b80 comp rx=0 tx=0).stop 2023-11-24T06:27:49.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 -- 172.21.15.114:0/2653681532 shutdown_connections 2023-11-24T06:27:49.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f670815e1b0 0x7f67080cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 --2- 172.21.15.114:0/2653681532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f67080d2240 0x7f67080b1680 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 -- 172.21.15.114:0/2653681532 >> 172.21.15.114:0/2653681532 conn(0x7f67080731a0 msgr2=0x7f670809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 -- 172.21.15.114:0/2653681532 shutdown_connections 2023-11-24T06:27:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.822+0000 7f670f9a6700 1 -- 172.21.15.114:0/2653681532 wait complete. 2023-11-24T06:27:49.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.823+0000 7f670f9a6700 1 Processor -- start 2023-11-24T06:27:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.823+0000 7f670f9a6700 1 -- start start 2023-11-24T06:27:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67080d2240 0x7f670814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f670815e1b0 0x7f67080b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670809dad0 con 0x7f670815ddd0 2023-11-24T06:27:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670809ddb0 con 0x7f670815e1b0 2023-11-24T06:27:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670809e090 con 0x7f67080d2240 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42204/0 (socket says 172.21.15.114:42204) 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 -- 172.21.15.114:0/1456446684 learned_addr learned my addr 172.21.15.114:0/1456446684 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670d742700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67080d2240 0x7f670814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 -- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67080d2240 msgr2=0x7f670814cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67080d2240 0x7f670814cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 -- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f670815e1b0 msgr2=0x7f67080b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f670815e1b0 0x7f67080b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 -- 172.21.15.114:0/1456446684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040094a0 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670cf41700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6704009840 tx=0x7f6704004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040036a0 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670401e070 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67080b2ed0 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.824+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704003a40 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.825+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670809e650 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.826+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708067f60 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.826+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6704003ba0 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.826+0000 7f66fdffb700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 0x7f66ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.827+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f670408a7f0 con 0x7f670815ddd0 2023-11-24T06:27:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.827+0000 7f670d742700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 0x7f66ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.827+0000 7f670d742700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 0x7f66ec06d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000f60 tx=0x7f66f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:49.831+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67040556b0 con 0x7f670815ddd0 2023-11-24T06:27:50.019 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.016+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67080b2550 con 0x7f66ec06b080 2023-11-24T06:27:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.024+0000 7f66fdffb700 1 -- 172.21.15.114:0/1456446684 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286422 (secure 0 0 0) 0x7f66f80070d0 con 0x7f66ec06b080 2023-11-24T06:27:50.031 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 msgr2=0x7f66ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 0x7f66ec06d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000f60 tx=0x7f66f8009040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 msgr2=0x7f67080b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6704009840 tx=0x7f6704004750 comp rx=0 tx=0).stop 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 shutdown_connections 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f66ec06b080 0x7f66ec06d540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f670815e1b0 0x7f67080b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67080d2240 0x7f670814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 --2- 172.21.15.114:0/1456446684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f670815ddd0 0x7f67080b5840 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 >> 172.21.15.114:0/1456446684 conn(0x7f67080731a0 msgr2=0x7f6708160680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 shutdown_connections 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.027+0000 7f670f9a6700 1 -- 172.21.15.114:0/1456446684 wait complete. 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:27:50.043 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-24T06:27:49.753268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222684,"kb_used_data":6500,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752036,"statfs":{"total":773094113280,"available":772866084864,"internally_reserved":0,"allocated":6656000,"data_stored":3429776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001141"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188600+0000","last_change":"2023-11-24T06:27:45.196764+0000","last_active":"2023-11-24T06:27:46.188600+0000","last_peered":"2023-11-24T06:27:46.188600+0000","last_clean":"2023-11-24T06:27:46.188600+0000","last_became_active":"2023-11-24T06:27:45.196459+0000","last_became_peered":"2023-11-24T06:27:45.196459+0000","last_unstale":"2023-11-24T06:27:46.188600+0000","last_undegraded":"2023-11-24T06:27:46.188600+0000","last_fullsized":"2023-11-24T06:27:46.188600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:10.953646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439357+0000","last_change":"2023-11-24T06:27:45.199441+0000","last_active":"2023-11-24T06:27:46.439357+0000","last_peered":"2023-11-24T06:27:46.439357+0000","last_clean":"2023-11-24T06:27:46.439357+0000","last_became_active":"2023-11-24T06:27:45.199159+0000","last_became_peered":"2023-11-24T06:27:45.199159+0000","last_unstale":"2023-11-24T06:27:46.439357+0000","last_undegraded":"2023-11-24T06:27:46.439357+0000","last_fullsized":"2023-11-24T06:27:46.439357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:47.888236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876797+0000","last_change":"2023-11-24T06:27:43.876797+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876797+0000","last_undegraded":"2023-11-24T06:27:43.876797+0000","last_fullsized":"2023-11-24T06:27:43.876797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880382+0000","last_change":"2023-11-24T06:27:43.880382+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880382+0000","last_undegraded":"2023-11-24T06:27:43.880382+0000","last_fullsized":"2023-11-24T06:27:43.880382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188111+0000","last_change":"2023-11-24T06:27:45.197360+0000","last_active":"2023-11-24T06:27:46.188111+0000","last_peered":"2023-11-24T06:27:46.188111+0000","last_clean":"2023-11-24T06:27:46.188111+0000","last_became_active":"2023-11-24T06:27:45.197045+0000","last_became_peered":"2023-11-24T06:27:45.197045+0000","last_unstale":"2023-11-24T06:27:46.188111+0000","last_undegraded":"2023-11-24T06:27:46.188111+0000","last_fullsized":"2023-11-24T06:27:46.188111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:50.810177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501275+0000","last_change":"2023-11-24T06:27:45.187750+0000","last_active":"2023-11-24T06:27:46.501275+0000","last_peered":"2023-11-24T06:27:46.501275+0000","last_clean":"2023-11-24T06:27:46.501275+0000","last_became_active":"2023-11-24T06:27:45.187445+0000","last_became_peered":"2023-11-24T06:27:45.187445+0000","last_unstale":"2023-11-24T06:27:46.501275+0000","last_undegraded":"2023-11-24T06:27:46.501275+0000","last_fullsized":"2023-11-24T06:27:46.501275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:06.517029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884229+0000","last_change":"2023-11-24T06:27:43.884229+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884229+0000","last_undegraded":"2023-11-24T06:27:43.884229+0000","last_fullsized":"2023-11-24T06:27:43.884229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439075+0000","last_change":"2023-11-24T06:27:45.193543+0000","last_active":"2023-11-24T06:27:46.439075+0000","last_peered":"2023-11-24T06:27:46.439075+0000","last_clean":"2023-11-24T06:27:46.439075+0000","last_became_active":"2023-11-24T06:27:45.192904+0000","last_became_peered":"2023-11-24T06:27:45.192904+0000","last_unstale":"2023-11-24T06:27:46.439075+0000","last_undegraded":"2023-11-24T06:27:46.439075+0000","last_fullsized":"2023-11-24T06:27:46.439075+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:47.980944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881743+0000","last_change":"2023-11-24T06:27:43.881743+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881743+0000","last_undegraded":"2023-11-24T06:27:43.881743+0000","last_fullsized":"2023-11-24T06:27:43.881743+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875949+0000","last_change":"2023-11-24T06:27:43.875949+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875949+0000","last_undegraded":"2023-11-24T06:27:43.875949+0000","last_fullsized":"2023-11-24T06:27:43.875949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876187+0000","last_change":"2023-11-24T06:27:43.876187+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876187+0000","last_undegraded":"2023-11-24T06:27:43.876187+0000","last_fullsized":"2023-11-24T06:27:43.876187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878485+0000","last_change":"2023-11-24T06:27:43.878485+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878485+0000","last_undegraded":"2023-11-24T06:27:43.878485+0000","last_fullsized":"2023-11-24T06:27:43.878485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874177+0000","last_change":"2023-11-24T06:27:43.874177+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874177+0000","last_undegraded":"2023-11-24T06:27:43.874177+0000","last_fullsized":"2023-11-24T06:27:43.874177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882087+0000","last_change":"2023-11-24T06:27:43.882087+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882087+0000","last_undegraded":"2023-11-24T06:27:43.882087+0000","last_fullsized":"2023-11-24T06:27:43.882087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883840+0000","last_change":"2023-11-24T06:27:43.883840+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883840+0000","last_undegraded":"2023-11-24T06:27:43.883840+0000","last_fullsized":"2023-11-24T06:27:43.883840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501084+0000","last_change":"2023-11-24T06:27:45.192899+0000","last_active":"2023-11-24T06:27:46.501084+0000","last_peered":"2023-11-24T06:27:46.501084+0000","last_clean":"2023-11-24T06:27:46.501084+0000","last_became_active":"2023-11-24T06:27:45.192662+0000","last_became_peered":"2023-11-24T06:27:45.192662+0000","last_unstale":"2023-11-24T06:27:46.501084+0000","last_undegraded":"2023-11-24T06:27:46.501084+0000","last_fullsized":"2023-11-24T06:27:46.501084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:28.618271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439465+0000","last_change":"2023-11-24T06:27:45.193370+0000","last_active":"2023-11-24T06:27:46.439465+0000","last_peered":"2023-11-24T06:27:46.439465+0000","last_clean":"2023-11-24T06:27:46.439465+0000","last_became_active":"2023-11-24T06:27:45.193080+0000","last_became_peered":"2023-11-24T06:27:45.193080+0000","last_unstale":"2023-11-24T06:27:46.439465+0000","last_undegraded":"2023-11-24T06:27:46.439465+0000","last_fullsized":"2023-11-24T06:27:46.439465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:46.645009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875034+0000","last_change":"2023-11-24T06:27:43.875034+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875034+0000","last_undegraded":"2023-11-24T06:27:43.875034+0000","last_fullsized":"2023-11-24T06:27:43.875034+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188370+0000","last_change":"2023-11-24T06:27:45.196827+0000","last_active":"2023-11-24T06:27:46.188370+0000","last_peered":"2023-11-24T06:27:46.188370+0000","last_clean":"2023-11-24T06:27:46.188370+0000","last_became_active":"2023-11-24T06:27:45.196474+0000","last_became_peered":"2023-11-24T06:27:45.196474+0000","last_unstale":"2023-11-24T06:27:46.188370+0000","last_undegraded":"2023-11-24T06:27:46.188370+0000","last_fullsized":"2023-11-24T06:27:46.188370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:36.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439320+0000","last_change":"2023-11-24T06:27:45.199695+0000","last_active":"2023-11-24T06:27:46.439320+0000","last_peered":"2023-11-24T06:27:46.439320+0000","last_clean":"2023-11-24T06:27:46.439320+0000","last_became_active":"2023-11-24T06:27:45.199046+0000","last_became_peered":"2023-11-24T06:27:45.199046+0000","last_unstale":"2023-11-24T06:27:46.439320+0000","last_undegraded":"2023-11-24T06:27:46.439320+0000","last_fullsized":"2023-11-24T06:27:46.439320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:02.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875751+0000","last_change":"2023-11-24T06:27:43.875751+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875751+0000","last_undegraded":"2023-11-24T06:27:43.875751+0000","last_fullsized":"2023-11-24T06:27:43.875751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439113+0000","last_change":"2023-11-24T06:27:45.196449+0000","last_active":"2023-11-24T06:27:46.439113+0000","last_peered":"2023-11-24T06:27:46.439113+0000","last_clean":"2023-11-24T06:27:46.439113+0000","last_became_active":"2023-11-24T06:27:45.196184+0000","last_became_peered":"2023-11-24T06:27:45.196184+0000","last_unstale":"2023-11-24T06:27:46.439113+0000","last_undegraded":"2023-11-24T06:27:46.439113+0000","last_fullsized":"2023-11-24T06:27:46.439113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:14.864903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873638+0000","last_change":"2023-11-24T06:27:43.873638+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873638+0000","last_undegraded":"2023-11-24T06:27:43.873638+0000","last_fullsized":"2023-11-24T06:27:43.873638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875332+0000","last_change":"2023-11-24T06:27:43.875332+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875332+0000","last_undegraded":"2023-11-24T06:27:43.875332+0000","last_fullsized":"2023-11-24T06:27:43.875332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188504+0000","last_change":"2023-11-24T06:27:45.199317+0000","last_active":"2023-11-24T06:27:46.188504+0000","last_peered":"2023-11-24T06:27:46.188504+0000","last_clean":"2023-11-24T06:27:46.188504+0000","last_became_active":"2023-11-24T06:27:45.198954+0000","last_became_peered":"2023-11-24T06:27:45.198954+0000","last_unstale":"2023-11-24T06:27:46.188504+0000","last_undegraded":"2023-11-24T06:27:46.188504+0000","last_fullsized":"2023-11-24T06:27:46.188504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:52.546554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878658+0000","last_change":"2023-11-24T06:27:43.878658+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878658+0000","last_undegraded":"2023-11-24T06:27:43.878658+0000","last_fullsized":"2023-11-24T06:27:43.878658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501543+0000","last_change":"2023-11-24T06:27:45.191845+0000","last_active":"2023-11-24T06:27:46.501543+0000","last_peered":"2023-11-24T06:27:46.501543+0000","last_clean":"2023-11-24T06:27:46.501543+0000","last_became_active":"2023-11-24T06:27:45.191565+0000","last_became_peered":"2023-11-24T06:27:45.191565+0000","last_unstale":"2023-11-24T06:27:46.501543+0000","last_undegraded":"2023-11-24T06:27:46.501543+0000","last_fullsized":"2023-11-24T06:27:46.501543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:10.419872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188231+0000","last_change":"2023-11-24T06:27:45.191652+0000","last_active":"2023-11-24T06:27:46.188231+0000","last_peered":"2023-11-24T06:27:46.188231+0000","last_clean":"2023-11-24T06:27:46.188231+0000","last_became_active":"2023-11-24T06:27:45.191360+0000","last_became_peered":"2023-11-24T06:27:45.191360+0000","last_unstale":"2023-11-24T06:27:46.188231+0000","last_undegraded":"2023-11-24T06:27:46.188231+0000","last_fullsized":"2023-11-24T06:27:46.188231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:17.745755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.191429+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.191159+0000","last_became_peered":"2023-11-24T06:27:45.191159+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.196091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874041+0000","last_change":"2023-11-24T06:27:43.874041+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874041+0000","last_undegraded":"2023-11-24T06:27:43.874041+0000","last_fullsized":"2023-11-24T06:27:43.874041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.187889+0000","last_change":"2023-11-24T06:27:45.199361+0000","last_active":"2023-11-24T06:27:46.187889+0000","last_peered":"2023-11-24T06:27:46.187889+0000","last_clean":"2023-11-24T06:27:46.187889+0000","last_became_active":"2023-11-24T06:27:45.199093+0000","last_became_peered":"2023-11-24T06:27:45.199093+0000","last_unstale":"2023-11-24T06:27:46.187889+0000","last_undegraded":"2023-11-24T06:27:46.187889+0000","last_fullsized":"2023-11-24T06:27:46.187889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:20.032382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501160+0000","last_change":"2023-11-24T06:27:45.194242+0000","last_active":"2023-11-24T06:27:46.501160+0000","last_peered":"2023-11-24T06:27:46.501160+0000","last_clean":"2023-11-24T06:27:46.501160+0000","last_became_active":"2023-11-24T06:27:45.193957+0000","last_became_peered":"2023-11-24T06:27:45.193957+0000","last_unstale":"2023-11-24T06:27:46.501160+0000","last_undegraded":"2023-11-24T06:27:46.501160+0000","last_fullsized":"2023-11-24T06:27:46.501160+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.755035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501812+0000","last_change":"2023-11-24T06:27:45.187022+0000","last_active":"2023-11-24T06:27:46.501812+0000","last_peered":"2023-11-24T06:27:46.501812+0000","last_clean":"2023-11-24T06:27:46.501812+0000","last_became_active":"2023-11-24T06:27:45.186746+0000","last_became_peered":"2023-11-24T06:27:45.186746+0000","last_unstale":"2023-11-24T06:27:46.501812+0000","last_undegraded":"2023-11-24T06:27:46.501812+0000","last_fullsized":"2023-11-24T06:27:46.501812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:07.653354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212303+0000","last_change":"2023-11-24T06:27:43.887983+0000","last_active":"2023-11-24T06:27:45.212303+0000","last_peered":"2023-11-24T06:27:45.212303+0000","last_clean":"2023-11-24T06:27:45.212303+0000","last_became_active":"2023-11-24T06:27:43.887687+0000","last_became_peered":"2023-11-24T06:27:43.887687+0000","last_unstale":"2023-11-24T06:27:45.212303+0000","last_undegraded":"2023-11-24T06:27:45.212303+0000","last_fullsized":"2023-11-24T06:27:45.212303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876827+0000","last_change":"2023-11-24T06:27:43.876827+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876827+0000","last_undegraded":"2023-11-24T06:27:43.876827+0000","last_fullsized":"2023-11-24T06:27:43.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207909+0000","last_change":"2023-11-24T06:27:43.887478+0000","last_active":"2023-11-24T06:27:45.207909+0000","last_peered":"2023-11-24T06:27:45.207909+0000","last_clean":"2023-11-24T06:27:45.207909+0000","last_became_active":"2023-11-24T06:27:43.886904+0000","last_became_peered":"2023-11-24T06:27:43.886904+0000","last_unstale":"2023-11-24T06:27:45.207909+0000","last_undegraded":"2023-11-24T06:27:45.207909+0000","last_fullsized":"2023-11-24T06:27:45.207909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:57.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501874+0000","last_change":"2023-11-24T06:27:45.184656+0000","last_active":"2023-11-24T06:27:46.501874+0000","last_peered":"2023-11-24T06:27:46.501874+0000","last_clean":"2023-11-24T06:27:46.501874+0000","last_became_active":"2023-11-24T06:27:45.184372+0000","last_became_peered":"2023-11-24T06:27:45.184372+0000","last_unstale":"2023-11-24T06:27:46.501874+0000","last_undegraded":"2023-11-24T06:27:46.501874+0000","last_fullsized":"2023-11-24T06:27:46.501874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:43.478543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209895+0000","last_change":"2023-11-24T06:27:43.893948+0000","last_active":"2023-11-24T06:27:45.209895+0000","last_peered":"2023-11-24T06:27:45.209895+0000","last_clean":"2023-11-24T06:27:45.209895+0000","last_became_active":"2023-11-24T06:27:43.893661+0000","last_became_peered":"2023-11-24T06:27:43.893661+0000","last_unstale":"2023-11-24T06:27:45.209895+0000","last_undegraded":"2023-11-24T06:27:45.209895+0000","last_fullsized":"2023-11-24T06:27:45.209895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:38:51.432906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501588+0000","last_change":"2023-11-24T06:27:45.189667+0000","last_active":"2023-11-24T06:27:46.501588+0000","last_peered":"2023-11-24T06:27:46.501588+0000","last_clean":"2023-11-24T06:27:46.501588+0000","last_became_active":"2023-11-24T06:27:45.189394+0000","last_became_peered":"2023-11-24T06:27:45.189394+0000","last_unstale":"2023-11-24T06:27:46.501588+0000","last_undegraded":"2023-11-24T06:27:46.501588+0000","last_fullsized":"2023-11-24T06:27:46.501588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:18.377171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887326+0000","last_change":"2023-11-24T06:27:43.884164+0000","last_active":"2023-11-24T06:27:43.887326+0000","last_peered":"2023-11-24T06:27:43.887326+0000","last_clean":"2023-11-24T06:27:43.887326+0000","last_became_active":"2023-11-24T06:27:43.883886+0000","last_became_peered":"2023-11-24T06:27:43.883886+0000","last_unstale":"2023-11-24T06:27:43.887326+0000","last_undegraded":"2023-11-24T06:27:43.887326+0000","last_fullsized":"2023-11-24T06:27:43.887326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:35.939429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439144+0000","last_change":"2023-11-24T06:27:45.196404+0000","last_active":"2023-11-24T06:27:46.439144+0000","last_peered":"2023-11-24T06:27:46.439144+0000","last_clean":"2023-11-24T06:27:46.439144+0000","last_became_active":"2023-11-24T06:27:45.196180+0000","last_became_peered":"2023-11-24T06:27:45.196180+0000","last_unstale":"2023-11-24T06:27:46.439144+0000","last_undegraded":"2023-11-24T06:27:46.439144+0000","last_fullsized":"2023-11-24T06:27:46.439144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:01.939068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891238+0000","last_change":"2023-11-24T06:27:43.887860+0000","last_active":"2023-11-24T06:27:43.891238+0000","last_peered":"2023-11-24T06:27:43.891238+0000","last_clean":"2023-11-24T06:27:43.891238+0000","last_became_active":"2023-11-24T06:27:43.887522+0000","last_became_peered":"2023-11-24T06:27:43.887522+0000","last_unstale":"2023-11-24T06:27:43.891238+0000","last_undegraded":"2023-11-24T06:27:43.891238+0000","last_fullsized":"2023-11-24T06:27:43.891238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:18.289207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439483+0000","last_change":"2023-11-24T06:27:45.190694+0000","last_active":"2023-11-24T06:27:46.439483+0000","last_peered":"2023-11-24T06:27:46.439483+0000","last_clean":"2023-11-24T06:27:46.439483+0000","last_became_active":"2023-11-24T06:27:45.190410+0000","last_became_peered":"2023-11-24T06:27:45.190410+0000","last_unstale":"2023-11-24T06:27:46.439483+0000","last_undegraded":"2023-11-24T06:27:46.439483+0000","last_fullsized":"2023-11-24T06:27:46.439483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:55.059669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208021+0000","last_change":"2023-11-24T06:27:43.888671+0000","last_active":"2023-11-24T06:27:45.208021+0000","last_peered":"2023-11-24T06:27:45.208021+0000","last_clean":"2023-11-24T06:27:45.208021+0000","last_became_active":"2023-11-24T06:27:43.888398+0000","last_became_peered":"2023-11-24T06:27:43.888398+0000","last_unstale":"2023-11-24T06:27:45.208021+0000","last_undegraded":"2023-11-24T06:27:45.208021+0000","last_fullsized":"2023-11-24T06:27:45.208021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:04.828359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873693+0000","last_change":"2023-11-24T06:27:43.873693+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873693+0000","last_undegraded":"2023-11-24T06:27:43.873693+0000","last_fullsized":"2023-11-24T06:27:43.873693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893351+0000","last_change":"2023-11-24T06:27:43.888597+0000","last_active":"2023-11-24T06:27:43.893351+0000","last_peered":"2023-11-24T06:27:43.893351+0000","last_clean":"2023-11-24T06:27:43.893351+0000","last_became_active":"2023-11-24T06:27:43.888055+0000","last_became_peered":"2023-11-24T06:27:43.888055+0000","last_unstale":"2023-11-24T06:27:43.893351+0000","last_undegraded":"2023-11-24T06:27:43.893351+0000","last_fullsized":"2023-11-24T06:27:43.893351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439166+0000","last_change":"2023-11-24T06:27:45.194260+0000","last_active":"2023-11-24T06:27:46.439166+0000","last_peered":"2023-11-24T06:27:46.439166+0000","last_clean":"2023-11-24T06:27:46.439166+0000","last_became_active":"2023-11-24T06:27:45.193685+0000","last_became_peered":"2023-11-24T06:27:45.193685+0000","last_unstale":"2023-11-24T06:27:46.439166+0000","last_undegraded":"2023-11-24T06:27:46.439166+0000","last_fullsized":"2023-11-24T06:27:46.439166+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:56.547263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893456+0000","last_change":"2023-11-24T06:27:43.890543+0000","last_active":"2023-11-24T06:27:43.893456+0000","last_peered":"2023-11-24T06:27:43.893456+0000","last_clean":"2023-11-24T06:27:43.893456+0000","last_became_active":"2023-11-24T06:27:43.890213+0000","last_became_peered":"2023-11-24T06:27:43.890213+0000","last_unstale":"2023-11-24T06:27:43.893456+0000","last_undegraded":"2023-11-24T06:27:43.893456+0000","last_fullsized":"2023-11-24T06:27:43.893456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:21.791667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881850+0000","last_change":"2023-11-24T06:27:43.881850+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881850+0000","last_undegraded":"2023-11-24T06:27:43.881850+0000","last_fullsized":"2023-11-24T06:27:43.881850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893344+0000","last_change":"2023-11-24T06:27:43.888862+0000","last_active":"2023-11-24T06:27:43.893344+0000","last_peered":"2023-11-24T06:27:43.893344+0000","last_clean":"2023-11-24T06:27:43.893344+0000","last_became_active":"2023-11-24T06:27:43.888560+0000","last_became_peered":"2023-11-24T06:27:43.888560+0000","last_unstale":"2023-11-24T06:27:43.893344+0000","last_undegraded":"2023-11-24T06:27:43.893344+0000","last_fullsized":"2023-11-24T06:27:43.893344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873668+0000","last_change":"2023-11-24T06:27:43.873668+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873668+0000","last_undegraded":"2023-11-24T06:27:43.873668+0000","last_fullsized":"2023-11-24T06:27:43.873668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.187762+0000","last_change":"2023-11-24T06:27:43.886888+0000","last_active":"2023-11-24T06:27:45.187762+0000","last_peered":"2023-11-24T06:27:45.187762+0000","last_clean":"2023-11-24T06:27:45.187762+0000","last_became_active":"2023-11-24T06:27:43.886584+0000","last_became_peered":"2023-11-24T06:27:43.886584+0000","last_unstale":"2023-11-24T06:27:45.187762+0000","last_undegraded":"2023-11-24T06:27:45.187762+0000","last_fullsized":"2023-11-24T06:27:45.187762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:17:04.535818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878276+0000","last_change":"2023-11-24T06:27:43.878276+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878276+0000","last_undegraded":"2023-11-24T06:27:43.878276+0000","last_fullsized":"2023-11-24T06:27:43.878276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891262+0000","last_change":"2023-11-24T06:27:43.887129+0000","last_active":"2023-11-24T06:27:43.891262+0000","last_peered":"2023-11-24T06:27:43.891262+0000","last_clean":"2023-11-24T06:27:43.891262+0000","last_became_active":"2023-11-24T06:27:43.886873+0000","last_became_peered":"2023-11-24T06:27:43.886873+0000","last_unstale":"2023-11-24T06:27:43.891262+0000","last_undegraded":"2023-11-24T06:27:43.891262+0000","last_fullsized":"2023-11-24T06:27:43.891262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875846+0000","last_change":"2023-11-24T06:27:43.875846+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875846+0000","last_undegraded":"2023-11-24T06:27:43.875846+0000","last_fullsized":"2023-11-24T06:27:43.875846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893615+0000","last_change":"2023-11-24T06:27:43.891037+0000","last_active":"2023-11-24T06:27:43.893615+0000","last_peered":"2023-11-24T06:27:43.893615+0000","last_clean":"2023-11-24T06:27:43.893615+0000","last_became_active":"2023-11-24T06:27:43.890756+0000","last_became_peered":"2023-11-24T06:27:43.890756+0000","last_unstale":"2023-11-24T06:27:43.893615+0000","last_undegraded":"2023-11-24T06:27:43.893615+0000","last_fullsized":"2023-11-24T06:27:43.893615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:56.949114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877698+0000","last_change":"2023-11-24T06:27:43.877698+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877698+0000","last_undegraded":"2023-11-24T06:27:43.877698+0000","last_fullsized":"2023-11-24T06:27:43.877698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209609+0000","last_change":"2023-11-24T06:27:43.887228+0000","last_active":"2023-11-24T06:27:45.209609+0000","last_peered":"2023-11-24T06:27:45.209609+0000","last_clean":"2023-11-24T06:27:45.209609+0000","last_became_active":"2023-11-24T06:27:43.886820+0000","last_became_peered":"2023-11-24T06:27:43.886820+0000","last_unstale":"2023-11-24T06:27:45.209609+0000","last_undegraded":"2023-11-24T06:27:45.209609+0000","last_fullsized":"2023-11-24T06:27:45.209609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:47.443503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875335+0000","last_change":"2023-11-24T06:27:43.875335+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875335+0000","last_undegraded":"2023-11-24T06:27:43.875335+0000","last_fullsized":"2023-11-24T06:27:43.875335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.881677+0000","last_change":"2023-11-24T06:27:43.880061+0000","last_active":"2023-11-24T06:27:43.881677+0000","last_peered":"2023-11-24T06:27:43.881677+0000","last_clean":"2023-11-24T06:27:43.881677+0000","last_became_active":"2023-11-24T06:27:43.879790+0000","last_became_peered":"2023-11-24T06:27:43.879790+0000","last_unstale":"2023-11-24T06:27:43.881677+0000","last_undegraded":"2023-11-24T06:27:43.881677+0000","last_fullsized":"2023-11-24T06:27:43.881677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:16.744160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881584+0000","last_change":"2023-11-24T06:27:43.881584+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881584+0000","last_undegraded":"2023-11-24T06:27:43.881584+0000","last_fullsized":"2023-11-24T06:27:43.881584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885337+0000","last_change":"2023-11-24T06:27:43.881814+0000","last_active":"2023-11-24T06:27:43.885337+0000","last_peered":"2023-11-24T06:27:43.885337+0000","last_clean":"2023-11-24T06:27:43.885337+0000","last_became_active":"2023-11-24T06:27:43.881477+0000","last_became_peered":"2023-11-24T06:27:43.881477+0000","last_unstale":"2023-11-24T06:27:43.885337+0000","last_undegraded":"2023-11-24T06:27:43.885337+0000","last_fullsized":"2023-11-24T06:27:43.885337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874261+0000","last_change":"2023-11-24T06:27:43.874261+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874261+0000","last_undegraded":"2023-11-24T06:27:43.874261+0000","last_fullsized":"2023-11-24T06:27:43.874261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.888453+0000","last_change":"2023-11-24T06:27:43.885895+0000","last_active":"2023-11-24T06:27:43.888453+0000","last_peered":"2023-11-24T06:27:43.888453+0000","last_clean":"2023-11-24T06:27:43.888453+0000","last_became_active":"2023-11-24T06:27:43.885623+0000","last_became_peered":"2023-11-24T06:27:43.885623+0000","last_unstale":"2023-11-24T06:27:43.888453+0000","last_undegraded":"2023-11-24T06:27:43.888453+0000","last_fullsized":"2023-11-24T06:27:43.888453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874451+0000","last_change":"2023-11-24T06:27:43.874451+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874451+0000","last_undegraded":"2023-11-24T06:27:43.874451+0000","last_fullsized":"2023-11-24T06:27:43.874451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209513+0000","last_change":"2023-11-24T06:27:43.887376+0000","last_active":"2023-11-24T06:27:45.209513+0000","last_peered":"2023-11-24T06:27:45.209513+0000","last_clean":"2023-11-24T06:27:45.209513+0000","last_became_active":"2023-11-24T06:27:43.887125+0000","last_became_peered":"2023-11-24T06:27:43.887125+0000","last_unstale":"2023-11-24T06:27:45.209513+0000","last_undegraded":"2023-11-24T06:27:45.209513+0000","last_fullsized":"2023-11-24T06:27:45.209513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:06.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878320+0000","last_change":"2023-11-24T06:27:43.878320+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878320+0000","last_undegraded":"2023-11-24T06:27:43.878320+0000","last_fullsized":"2023-11-24T06:27:43.878320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208265+0000","last_change":"2023-11-24T06:27:43.887382+0000","last_active":"2023-11-24T06:27:45.208265+0000","last_peered":"2023-11-24T06:27:45.208265+0000","last_clean":"2023-11-24T06:27:45.208265+0000","last_became_active":"2023-11-24T06:27:43.887150+0000","last_became_peered":"2023-11-24T06:27:43.887150+0000","last_unstale":"2023-11-24T06:27:45.208265+0000","last_undegraded":"2023-11-24T06:27:45.208265+0000","last_fullsized":"2023-11-24T06:27:45.208265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.841385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.197917+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.197646+0000","last_became_peered":"2023-11-24T06:27:45.197646+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:12.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887186+0000","last_change":"2023-11-24T06:27:43.884248+0000","last_active":"2023-11-24T06:27:43.887186+0000","last_peered":"2023-11-24T06:27:43.887186+0000","last_clean":"2023-11-24T06:27:43.887186+0000","last_became_active":"2023-11-24T06:27:43.883575+0000","last_became_peered":"2023-11-24T06:27:43.883575+0000","last_unstale":"2023-11-24T06:27:43.887186+0000","last_undegraded":"2023-11-24T06:27:43.887186+0000","last_fullsized":"2023-11-24T06:27:43.887186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:17:46.473315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874503+0000","last_change":"2023-11-24T06:27:43.874503+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874503+0000","last_undegraded":"2023-11-24T06:27:43.874503+0000","last_fullsized":"2023-11-24T06:27:43.874503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179085+0000","last_change":"2023-11-24T06:27:43.890800+0000","last_active":"2023-11-24T06:27:45.179085+0000","last_peered":"2023-11-24T06:27:45.179085+0000","last_clean":"2023-11-24T06:27:45.179085+0000","last_became_active":"2023-11-24T06:27:43.890508+0000","last_became_peered":"2023-11-24T06:27:43.890508+0000","last_unstale":"2023-11-24T06:27:45.179085+0000","last_undegraded":"2023-11-24T06:27:45.179085+0000","last_fullsized":"2023-11-24T06:27:45.179085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:04.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877020+0000","last_change":"2023-11-24T06:27:43.877020+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877020+0000","last_undegraded":"2023-11-24T06:27:43.877020+0000","last_fullsized":"2023-11-24T06:27:43.877020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212410+0000","last_change":"2023-11-24T06:27:43.888453+0000","last_active":"2023-11-24T06:27:45.212410+0000","last_peered":"2023-11-24T06:27:45.212410+0000","last_clean":"2023-11-24T06:27:45.212410+0000","last_became_active":"2023-11-24T06:27:43.888169+0000","last_became_peered":"2023-11-24T06:27:43.888169+0000","last_unstale":"2023-11-24T06:27:45.212410+0000","last_undegraded":"2023-11-24T06:27:45.212410+0000","last_fullsized":"2023-11-24T06:27:45.212410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:37.661869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501526+0000","last_change":"2023-11-24T06:27:45.192208+0000","last_active":"2023-11-24T06:27:46.501526+0000","last_peered":"2023-11-24T06:27:46.501526+0000","last_clean":"2023-11-24T06:27:46.501526+0000","last_became_active":"2023-11-24T06:27:45.191906+0000","last_became_peered":"2023-11-24T06:27:45.191906+0000","last_unstale":"2023-11-24T06:27:46.501526+0000","last_undegraded":"2023-11-24T06:27:46.501526+0000","last_fullsized":"2023-11-24T06:27:46.501526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:28.930496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893367+0000","last_change":"2023-11-24T06:27:43.886183+0000","last_active":"2023-11-24T06:27:43.893367+0000","last_peered":"2023-11-24T06:27:43.893367+0000","last_clean":"2023-11-24T06:27:43.893367+0000","last_became_active":"2023-11-24T06:27:43.885892+0000","last_became_peered":"2023-11-24T06:27:43.885892+0000","last_unstale":"2023-11-24T06:27:43.893367+0000","last_undegraded":"2023-11-24T06:27:43.893367+0000","last_fullsized":"2023-11-24T06:27:43.893367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:09.952050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501311+0000","last_change":"2023-11-24T06:27:45.187149+0000","last_active":"2023-11-24T06:27:46.501311+0000","last_peered":"2023-11-24T06:27:46.501311+0000","last_clean":"2023-11-24T06:27:46.501311+0000","last_became_active":"2023-11-24T06:27:45.186822+0000","last_became_peered":"2023-11-24T06:27:45.186822+0000","last_unstale":"2023-11-24T06:27:46.501311+0000","last_undegraded":"2023-11-24T06:27:46.501311+0000","last_fullsized":"2023-11-24T06:27:46.501311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:56.389229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893378+0000","last_change":"2023-11-24T06:27:43.890788+0000","last_active":"2023-11-24T06:27:43.893378+0000","last_peered":"2023-11-24T06:27:43.893378+0000","last_clean":"2023-11-24T06:27:43.893378+0000","last_became_active":"2023-11-24T06:27:43.890453+0000","last_became_peered":"2023-11-24T06:27:43.890453+0000","last_unstale":"2023-11-24T06:27:43.893378+0000","last_undegraded":"2023-11-24T06:27:43.893378+0000","last_fullsized":"2023-11-24T06:27:43.893378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877120+0000","last_change":"2023-11-24T06:27:43.877120+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877120+0000","last_undegraded":"2023-11-24T06:27:43.877120+0000","last_fullsized":"2023-11-24T06:27:43.877120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893473+0000","last_change":"2023-11-24T06:27:43.889734+0000","last_active":"2023-11-24T06:27:43.893473+0000","last_peered":"2023-11-24T06:27:43.893473+0000","last_clean":"2023-11-24T06:27:43.893473+0000","last_became_active":"2023-11-24T06:27:43.889470+0000","last_became_peered":"2023-11-24T06:27:43.889470+0000","last_unstale":"2023-11-24T06:27:43.893473+0000","last_undegraded":"2023-11-24T06:27:43.893473+0000","last_fullsized":"2023-11-24T06:27:43.893473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:45.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877591+0000","last_change":"2023-11-24T06:27:43.877591+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877591+0000","last_undegraded":"2023-11-24T06:27:43.877591+0000","last_fullsized":"2023-11-24T06:27:43.877591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501897+0000","last_change":"2023-11-24T06:27:43.889511+0000","last_active":"2023-11-24T06:27:46.501897+0000","last_peered":"2023-11-24T06:27:46.501897+0000","last_clean":"2023-11-24T06:27:46.501897+0000","last_became_active":"2023-11-24T06:27:43.889234+0000","last_became_peered":"2023-11-24T06:27:43.889234+0000","last_unstale":"2023-11-24T06:27:46.501897+0000","last_undegraded":"2023-11-24T06:27:46.501897+0000","last_fullsized":"2023-11-24T06:27:46.501897+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:01.413877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875653+0000","last_change":"2023-11-24T06:27:43.875653+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875653+0000","last_undegraded":"2023-11-24T06:27:43.875653+0000","last_fullsized":"2023-11-24T06:27:43.875653+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887987+0000","last_change":"2023-11-24T06:27:43.886417+0000","last_active":"2023-11-24T06:27:43.887987+0000","last_peered":"2023-11-24T06:27:43.887987+0000","last_clean":"2023-11-24T06:27:43.887987+0000","last_became_active":"2023-11-24T06:27:43.886110+0000","last_became_peered":"2023-11-24T06:27:43.886110+0000","last_unstale":"2023-11-24T06:27:43.887987+0000","last_undegraded":"2023-11-24T06:27:43.887987+0000","last_fullsized":"2023-11-24T06:27:43.887987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:35.558430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501695+0000","last_change":"2023-11-24T06:27:45.184597+0000","last_active":"2023-11-24T06:27:46.501695+0000","last_peered":"2023-11-24T06:27:46.501695+0000","last_clean":"2023-11-24T06:27:46.501695+0000","last_became_active":"2023-11-24T06:27:45.184299+0000","last_became_peered":"2023-11-24T06:27:45.184299+0000","last_unstale":"2023-11-24T06:27:46.501695+0000","last_undegraded":"2023-11-24T06:27:46.501695+0000","last_fullsized":"2023-11-24T06:27:46.501695+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:51.039635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891002+0000","last_change":"2023-11-24T06:27:43.887202+0000","last_active":"2023-11-24T06:27:43.891002+0000","last_peered":"2023-11-24T06:27:43.891002+0000","last_clean":"2023-11-24T06:27:43.891002+0000","last_became_active":"2023-11-24T06:27:43.886926+0000","last_became_peered":"2023-11-24T06:27:43.886926+0000","last_unstale":"2023-11-24T06:27:43.891002+0000","last_undegraded":"2023-11-24T06:27:43.891002+0000","last_fullsized":"2023-11-24T06:27:43.891002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439423+0000","last_change":"2023-11-24T06:27:45.199563+0000","last_active":"2023-11-24T06:27:46.439423+0000","last_peered":"2023-11-24T06:27:46.439423+0000","last_clean":"2023-11-24T06:27:46.439423+0000","last_became_active":"2023-11-24T06:27:45.199267+0000","last_became_peered":"2023-11-24T06:27:45.199267+0000","last_unstale":"2023-11-24T06:27:46.439423+0000","last_undegraded":"2023-11-24T06:27:46.439423+0000","last_fullsized":"2023-11-24T06:27:46.439423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:45.747801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893321+0000","last_change":"2023-11-24T06:27:43.885639+0000","last_active":"2023-11-24T06:27:43.893321+0000","last_peered":"2023-11-24T06:27:43.893321+0000","last_clean":"2023-11-24T06:27:43.893321+0000","last_became_active":"2023-11-24T06:27:43.885321+0000","last_became_peered":"2023-11-24T06:27:43.885321+0000","last_unstale":"2023-11-24T06:27:43.893321+0000","last_undegraded":"2023-11-24T06:27:43.893321+0000","last_fullsized":"2023-11-24T06:27:43.893321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874894+0000","last_change":"2023-11-24T06:27:43.874894+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874894+0000","last_undegraded":"2023-11-24T06:27:43.874894+0000","last_fullsized":"2023-11-24T06:27:43.874894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893401+0000","last_change":"2023-11-24T06:27:43.888919+0000","last_active":"2023-11-24T06:27:43.893401+0000","last_peered":"2023-11-24T06:27:43.893401+0000","last_clean":"2023-11-24T06:27:43.893401+0000","last_became_active":"2023-11-24T06:27:43.888593+0000","last_became_peered":"2023-11-24T06:27:43.888593+0000","last_unstale":"2023-11-24T06:27:43.893401+0000","last_undegraded":"2023-11-24T06:27:43.893401+0000","last_fullsized":"2023-11-24T06:27:43.893401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:01.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875691+0000","last_change":"2023-11-24T06:27:43.875691+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875691+0000","last_undegraded":"2023-11-24T06:27:43.875691+0000","last_fullsized":"2023-11-24T06:27:43.875691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893416+0000","last_change":"2023-11-24T06:27:43.889027+0000","last_active":"2023-11-24T06:27:43.893416+0000","last_peered":"2023-11-24T06:27:43.893416+0000","last_clean":"2023-11-24T06:27:43.893416+0000","last_became_active":"2023-11-24T06:27:43.888756+0000","last_became_peered":"2023-11-24T06:27:43.888756+0000","last_unstale":"2023-11-24T06:27:43.893416+0000","last_undegraded":"2023-11-24T06:27:43.893416+0000","last_fullsized":"2023-11-24T06:27:43.893416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:59.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439231+0000","last_change":"2023-11-24T06:27:45.194043+0000","last_active":"2023-11-24T06:27:46.439231+0000","last_peered":"2023-11-24T06:27:46.439231+0000","last_clean":"2023-11-24T06:27:46.439231+0000","last_became_active":"2023-11-24T06:27:45.192605+0000","last_became_peered":"2023-11-24T06:27:45.192605+0000","last_unstale":"2023-11-24T06:27:46.439231+0000","last_undegraded":"2023-11-24T06:27:46.439231+0000","last_fullsized":"2023-11-24T06:27:46.439231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:59.614993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.180294+0000","last_change":"2023-11-24T06:27:43.889915+0000","last_active":"2023-11-24T06:27:45.180294+0000","last_peered":"2023-11-24T06:27:45.180294+0000","last_clean":"2023-11-24T06:27:45.180294+0000","last_became_active":"2023-11-24T06:27:43.889576+0000","last_became_peered":"2023-11-24T06:27:43.889576+0000","last_unstale":"2023-11-24T06:27:45.180294+0000","last_undegraded":"2023-11-24T06:27:45.180294+0000","last_fullsized":"2023-11-24T06:27:45.180294+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:41:42.721820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439124+0000","last_change":"2023-11-24T06:27:45.193323+0000","last_active":"2023-11-24T06:27:46.439124+0000","last_peered":"2023-11-24T06:27:46.439124+0000","last_clean":"2023-11-24T06:27:46.439124+0000","last_became_active":"2023-11-24T06:27:45.192548+0000","last_became_peered":"2023-11-24T06:27:45.192548+0000","last_unstale":"2023-11-24T06:27:46.439124+0000","last_undegraded":"2023-11-24T06:27:46.439124+0000","last_fullsized":"2023-11-24T06:27:46.439124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:09.532953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885063+0000","last_change":"2023-11-24T06:27:43.882518+0000","last_active":"2023-11-24T06:27:43.885063+0000","last_peered":"2023-11-24T06:27:43.885063+0000","last_clean":"2023-11-24T06:27:43.885063+0000","last_became_active":"2023-11-24T06:27:43.882224+0000","last_became_peered":"2023-11-24T06:27:43.882224+0000","last_unstale":"2023-11-24T06:27:43.885063+0000","last_undegraded":"2023-11-24T06:27:43.885063+0000","last_fullsized":"2023-11-24T06:27:43.885063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.872822+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:27:42.872822+0000","last_peered":"2023-11-24T06:27:42.872822+0000","last_clean":"2023-11-24T06:27:42.872822+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:27:42.872822+0000","last_undegraded":"2023-11-24T06:27:42.872822+0000","last_fullsized":"2023-11-24T06:27:42.872822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.77600000000000002,"15min":0.86599999999999999},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.87,"5min":1.373,"15min":1.373},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.411},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.345,"15min":1.345},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.80700000000000005,"5min":1.286,"15min":1.286},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46800000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":1.379,"15min":1.379},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.37},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.27000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.307}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.57799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.315,"15min":0.313},"min":{"1min":0.20899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.254},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.29699999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51200000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":1.028,"15min":1.028},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91600000000000004,"5min":1.01,"15min":1.01},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.0369999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":1,"15min":1},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.316},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.751},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.496}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.377},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:50.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-11-24T06:27:50.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.876797+0000', 'last_change': '2023-11-24T06:27:43.876797+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.876797+0000', 'last_undegraded': '2023-11-24T06:27:43.876797+0000', 'last_fullsized': '2023-11-24T06:27:43.876797+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.880382+0000', 'last_change': '2023-11-24T06:27:43.880382+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.880382+0000', 'last_undegraded': '2023-11-24T06:27:43.880382+0000', 'last_fullsized': '2023-11-24T06:27:43.880382+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.884229+0000', 'last_change': '2023-11-24T06:27:43.884229+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.884229+0000', 'last_undegraded': '2023-11-24T06:27:43.884229+0000', 'last_fullsized': '2023-11-24T06:27:43.884229+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.881743+0000', 'last_change': '2023-11-24T06:27:43.881743+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.881743+0000', 'last_undegraded': '2023-11-24T06:27:43.881743+0000', 'last_fullsized': '2023-11-24T06:27:43.881743+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875949+0000', 'last_change': '2023-11-24T06:27:43.875949+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875949+0000', 'last_undegraded': '2023-11-24T06:27:43.875949+0000', 'last_fullsized': '2023-11-24T06:27:43.875949+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-11-24T06:27:50.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.876187+0000', 'last_change': '2023-11-24T06:27:43.876187+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.876187+0000', 'last_undegraded': '2023-11-24T06:27:43.876187+0000', 'last_fullsized': '2023-11-24T06:27:43.876187+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.878485+0000', 'last_change': '2023-11-24T06:27:43.878485+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.878485+0000', 'last_undegraded': '2023-11-24T06:27:43.878485+0000', 'last_fullsized': '2023-11-24T06:27:43.878485+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874177+0000', 'last_change': '2023-11-24T06:27:43.874177+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874177+0000', 'last_undegraded': '2023-11-24T06:27:43.874177+0000', 'last_fullsized': '2023-11-24T06:27:43.874177+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.882087+0000', 'last_change': '2023-11-24T06:27:43.882087+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.882087+0000', 'last_undegraded': '2023-11-24T06:27:43.882087+0000', 'last_fullsized': '2023-11-24T06:27:43.882087+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.883840+0000', 'last_change': '2023-11-24T06:27:43.883840+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.883840+0000', 'last_undegraded': '2023-11-24T06:27:43.883840+0000', 'last_fullsized': '2023-11-24T06:27:43.883840+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875034+0000', 'last_change': '2023-11-24T06:27:43.875034+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875034+0000', 'last_undegraded': '2023-11-24T06:27:43.875034+0000', 'last_fullsized': '2023-11-24T06:27:43.875034+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875751+0000', 'last_change': '2023-11-24T06:27:43.875751+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875751+0000', 'last_undegraded': '2023-11-24T06:27:43.875751+0000', 'last_fullsized': '2023-11-24T06:27:43.875751+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.873638+0000', 'last_change': '2023-11-24T06:27:43.873638+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.873638+0000', 'last_undegraded': '2023-11-24T06:27:43.873638+0000', 'last_fullsized': '2023-11-24T06:27:43.873638+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875332+0000', 'last_change': '2023-11-24T06:27:43.875332+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875332+0000', 'last_undegraded': '2023-11-24T06:27:43.875332+0000', 'last_fullsized': '2023-11-24T06:27:43.875332+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-11-24T06:27:50.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.878658+0000', 'last_change': '2023-11-24T06:27:43.878658+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.878658+0000', 'last_undegraded': '2023-11-24T06:27:43.878658+0000', 'last_fullsized': '2023-11-24T06:27:43.878658+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874041+0000', 'last_change': '2023-11-24T06:27:43.874041+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874041+0000', 'last_undegraded': '2023-11-24T06:27:43.874041+0000', 'last_fullsized': '2023-11-24T06:27:43.874041+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.876827+0000', 'last_change': '2023-11-24T06:27:43.876827+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.876827+0000', 'last_undegraded': '2023-11-24T06:27:43.876827+0000', 'last_fullsized': '2023-11-24T06:27:43.876827+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.873693+0000', 'last_change': '2023-11-24T06:27:43.873693+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.873693+0000', 'last_undegraded': '2023-11-24T06:27:43.873693+0000', 'last_fullsized': '2023-11-24T06:27:43.873693+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.881850+0000', 'last_change': '2023-11-24T06:27:43.881850+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.881850+0000', 'last_undegraded': '2023-11-24T06:27:43.881850+0000', 'last_fullsized': '2023-11-24T06:27:43.881850+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-24T06:27:50.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.873668+0000', 'last_change': '2023-11-24T06:27:43.873668+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.873668+0000', 'last_undegraded': '2023-11-24T06:27:43.873668+0000', 'last_fullsized': '2023-11-24T06:27:43.873668+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.878276+0000', 'last_change': '2023-11-24T06:27:43.878276+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.878276+0000', 'last_undegraded': '2023-11-24T06:27:43.878276+0000', 'last_fullsized': '2023-11-24T06:27:43.878276+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875846+0000', 'last_change': '2023-11-24T06:27:43.875846+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875846+0000', 'last_undegraded': '2023-11-24T06:27:43.875846+0000', 'last_fullsized': '2023-11-24T06:27:43.875846+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.877698+0000', 'last_change': '2023-11-24T06:27:43.877698+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.877698+0000', 'last_undegraded': '2023-11-24T06:27:43.877698+0000', 'last_fullsized': '2023-11-24T06:27:43.877698+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875335+0000', 'last_change': '2023-11-24T06:27:43.875335+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875335+0000', 'last_undegraded': '2023-11-24T06:27:43.875335+0000', 'last_fullsized': '2023-11-24T06:27:43.875335+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-24T06:27:50.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.881584+0000', 'last_change': '2023-11-24T06:27:43.881584+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.881584+0000', 'last_undegraded': '2023-11-24T06:27:43.881584+0000', 'last_fullsized': '2023-11-24T06:27:43.881584+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:27:50.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-24T06:27:50.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874261+0000', 'last_change': '2023-11-24T06:27:43.874261+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874261+0000', 'last_undegraded': '2023-11-24T06:27:43.874261+0000', 'last_fullsized': '2023-11-24T06:27:43.874261+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874451+0000', 'last_change': '2023-11-24T06:27:43.874451+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874451+0000', 'last_undegraded': '2023-11-24T06:27:43.874451+0000', 'last_fullsized': '2023-11-24T06:27:43.874451+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.878320+0000', 'last_change': '2023-11-24T06:27:43.878320+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.878320+0000', 'last_undegraded': '2023-11-24T06:27:43.878320+0000', 'last_fullsized': '2023-11-24T06:27:43.878320+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874503+0000', 'last_change': '2023-11-24T06:27:43.874503+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874503+0000', 'last_undegraded': '2023-11-24T06:27:43.874503+0000', 'last_fullsized': '2023-11-24T06:27:43.874503+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.877020+0000', 'last_change': '2023-11-24T06:27:43.877020+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.877020+0000', 'last_undegraded': '2023-11-24T06:27:43.877020+0000', 'last_fullsized': '2023-11-24T06:27:43.877020+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-24T06:27:50.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.877120+0000', 'last_change': '2023-11-24T06:27:43.877120+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.877120+0000', 'last_undegraded': '2023-11-24T06:27:43.877120+0000', 'last_fullsized': '2023-11-24T06:27:43.877120+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.877591+0000', 'last_change': '2023-11-24T06:27:43.877591+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.877591+0000', 'last_undegraded': '2023-11-24T06:27:43.877591+0000', 'last_fullsized': '2023-11-24T06:27:43.877591+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875653+0000', 'last_change': '2023-11-24T06:27:43.875653+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875653+0000', 'last_undegraded': '2023-11-24T06:27:43.875653+0000', 'last_fullsized': '2023-11-24T06:27:43.875653+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.874894+0000', 'last_change': '2023-11-24T06:27:43.874894+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.874894+0000', 'last_undegraded': '2023-11-24T06:27:43.874894+0000', 'last_fullsized': '2023-11-24T06:27:43.874894+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-24T06:27:50.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:27:43.875691+0000', 'last_change': '2023-11-24T06:27:43.875691+0000', 'last_active': '2023-11-24T06:27:43.868182+0000', 'last_peered': '2023-11-24T06:27:43.868182+0000', 'last_clean': '2023-11-24T06:27:43.868182+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:27:43.875691+0000', 'last_undegraded': '2023-11-24T06:27:43.875691+0000', 'last_fullsized': '2023-11-24T06:27:43.875691+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'last_clean_scrub_stamp': '2023-11-24T06:27:43.868182+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:27:50.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T06:27:50.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.201+0000 7fb43c3b0700 1 Processor -- start 2023-11-24T06:27:50.205 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.201+0000 7fb43c3b0700 1 -- start start 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43c3b0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 0x7fb4340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43c3b0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4340af4d0 0x7fb4340afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43c3b0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4340ae430 con 0x7fb43406c000 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43c3b0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340af0b0 con 0x7fb4340af4d0 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43c3b0700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340b05a0 con 0x7fb43415e080 2023-11-24T06:27:50.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43994b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4340af4d0 0x7fb4340afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43a94d700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb43406c000 0x7fb434073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57920/0 (socket says 172.21.15.114:57920) 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43a94d700 1 -- 172.21.15.114:0/2446243075 learned_addr learned my addr 172.21.15.114:0/2446243075 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.202+0000 7fb43a14c700 1 --2- 172.21.15.114:0/2446243075 >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 0x7fb4340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43994b700 1 -- 172.21.15.114:0/2446243075 >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 msgr2=0x7fb4340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43994b700 1 --2- 172.21.15.114:0/2446243075 >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 0x7fb4340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43994b700 1 -- 172.21.15.114:0/2446243075 >> v1:172.21.15.114:6789/0 conn(0x7fb43406c000 legacy=0x7fb434073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43994b700 1 -- 172.21.15.114:0/2446243075 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340ae710 con 0x7fb4340af4d0 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43a14c700 1 --2- 172.21.15.114:0/2446243075 >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 0x7fb4340b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:50.207 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.203+0000 7fb43994b700 1 --2- 172.21.15.114:0/2446243075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb4340afb90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb434072a90 tx=0x7fb424009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dec547bf58e3cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43914a700 1 -- 172.21.15.114:0/2446243075 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424011830 con 0x7fb4340af4d0 2023-11-24T06:27:50.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43914a700 1 -- 172.21.15.114:0/2446243075 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb424011990 con 0x7fb4340af4d0 2023-11-24T06:27:50.208 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43914a700 1 -- 172.21.15.114:0/2446243075 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424011d20 con 0x7fb4340af4d0 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2446243075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 msgr2=0x7fb4340afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2446243075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb4340afb90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb434072a90 tx=0x7fb424009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2446243075 shutdown_connections 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2446243075 >> v2:172.21.15.172:3300/0 conn(0x7fb43415e080 0x7fb4340b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2446243075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb4340afb90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.204+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2446243075 >> 172.21.15.114:0/2446243075 conn(0x7fb434071b90 msgr2=0x7fb434071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.205+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2446243075 shutdown_connections 2023-11-24T06:27:50.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.205+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2446243075 wait complete. 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.206+0000 7fb43c3b0700 1 Processor -- start 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.206+0000 7fb43c3b0700 1 -- start start 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.206+0000 7fb43c3b0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb43406c000 0x7fb434155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.206+0000 7fb43c3b0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.206+0000 7fb43c3b0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb43415e080 0x7fb434147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43c3b0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434158000 con 0x7fb4340af4d0 2023-11-24T06:27:50.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43c3b0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434147ac0 con 0x7fb43415e080 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43c3b0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341581f0 con 0x7fb43406c000 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42218/0 (socket says 172.21.15.114:42218) 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 -- 172.21.15.114:0/2832771246 learned_addr learned my addr 172.21.15.114:0/2832771246 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43a94d700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb43415e080 0x7fb434147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43a14c700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb43406c000 0x7fb434155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 -- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb43406c000 msgr2=0x7fb434155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb43406c000 0x7fb434155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 -- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb43415e080 msgr2=0x7fb434147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.212 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb43415e080 0x7fb434147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 -- 172.21.15.114:0/2832771246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240094a0 con 0x7fb4340af4d0 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43994b700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097c0 tx=0x7fb424004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240036a0 con 0x7fb4340af4d0 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42401f070 con 0x7fb4340af4d0 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424003a40 con 0x7fb4340af4d0 2023-11-24T06:27:50.213 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4341584d0 con 0x7fb4340af4d0 2023-11-24T06:27:50.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.207+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb434158a90 con 0x7fb4340af4d0 2023-11-24T06:27:50.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.208+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4340753f0 con 0x7fb4340af4d0 2023-11-24T06:27:50.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.209+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb424003ba0 con 0x7fb4340af4d0 2023-11-24T06:27:50.214 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.209+0000 7fb42affd700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.209+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb424021070 con 0x7fb4340af4d0 2023-11-24T06:27:50.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.209+0000 7fb43a14c700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.210+0000 7fb43a14c700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 0x7fb41806d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c900 tx=0x7fb430008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.214+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb424056120 con 0x7fb4340af4d0 2023-11-24T06:27:50.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.487+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb43406faa0 con 0x7fb4340af4d0 2023-11-24T06:27:50.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.487+0000 7fb42affd700 1 -- 172.21.15.114:0/2832771246 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7fb424059740 con 0x7fb4340af4d0 2023-11-24T06:27:50.491 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:50.491 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1700807043,"num_in_osds":8,"osd_in_since":1700807034,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":62},{"state_name":"creating+peering","count":35}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591006,"bytes_used":228028416,"bytes_avail":772866084864,"bytes_total":773094113280,"inactive_pgs_ratio":0.36082473397254944,"write_bytes_sec":1535,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":37,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6216}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-24T06:26:57.741627+0000","services":{}},"progress_events":{}} 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 msgr2=0x7fb41806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 0x7fb41806d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c900 tx=0x7fb430008040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 msgr2=0x7fb434157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097c0 tx=0x7fb424004750 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 shutdown_connections 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb41806b080 0x7fb41806d540 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb43415e080 0x7fb434147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb43406c000 0x7fb434155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 --2- 172.21.15.114:0/2832771246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4340af4d0 0x7fb434157ac0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 >> 172.21.15.114:0/2832771246 conn(0x7fb434071b90 msgr2=0x7fb434160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 shutdown_connections 2023-11-24T06:27:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.489+0000 7fb43c3b0700 1 -- 172.21.15.114:0/2832771246 wait complete. 2023-11-24T06:27:50.504 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T06:27:50.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:50.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.621+0000 7f97c3469700 1 Processor -- start 2023-11-24T06:27:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.621+0000 7f97c3469700 1 -- start start 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c3469700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97bc0d2380 0x7f97bc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c3469700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc0b22d0 con 0x7f97bc0b3370 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c3469700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97bc0b25b0 con 0x7f97bc15bdd0 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c3469700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97bc0b2f50 con 0x7f97bc0d2380 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c1a06700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97bc0d2380 0x7f97bc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c1a06700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f97bc0d2380 0x7f97bc1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42228/0 (socket says 172.21.15.114:42228) 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c1a06700 1 -- 172.21.15.114:0/325761557 learned_addr learned my addr 172.21.15.114:0/325761557 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.622+0000 7f97c1a06700 1 -- 172.21.15.114:0/325761557 >> v1:172.21.15.172:6789/0 conn(0x7f97bc15bdd0 legacy=0x7f97bc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c1a06700 1 -- 172.21.15.114:0/325761557 >> v1:172.21.15.114:6789/0 conn(0x7f97bc0b3370 legacy=0x7f97bc0d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c1a06700 1 -- 172.21.15.114:0/325761557 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0b2890 con 0x7f97bc0d2380 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c1a06700 1 --2- 172.21.15.114:0/325761557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc0d2380 0x7f97bc1604e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009e50 tx=0x7f97b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fbf912c34def3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97b3fff700 1 -- 172.21.15.114:0/325761557 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400e070 con 0x7f97bc0d2380 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97b3fff700 1 -- 172.21.15.114:0/325761557 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b400b9c0 con 0x7f97bc0d2380 2023-11-24T06:27:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97b3fff700 1 -- 172.21.15.114:0/325761557 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400bd70 con 0x7f97bc0d2380 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c3469700 1 -- 172.21.15.114:0/325761557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc0d2380 msgr2=0x7f97bc1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c3469700 1 --2- 172.21.15.114:0/325761557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc0d2380 0x7f97bc1604e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009e50 tx=0x7f97b400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c3469700 1 -- 172.21.15.114:0/325761557 shutdown_connections 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c3469700 1 --2- 172.21.15.114:0/325761557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc0d2380 0x7f97bc1604e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.623+0000 7f97c3469700 1 -- 172.21.15.114:0/325761557 >> 172.21.15.114:0/325761557 conn(0x7f97bc0731a0 msgr2=0x7f97bc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.624+0000 7f97c3469700 1 -- 172.21.15.114:0/325761557 shutdown_connections 2023-11-24T06:27:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.624+0000 7f97c3469700 1 -- 172.21.15.114:0/325761557 wait complete. 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 Processor -- start 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 -- start start 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97bc0b3370 0x7f97bc0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 0x7f97bc0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0c9600 con 0x7f97bc15bdd0 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0c9320 con 0x7f97bc0b3370 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.625+0000 7f97c3469700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc0cc820 con 0x7f97bc0d2380 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42234/0 (socket says 172.21.15.114:42234) 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 -- 172.21.15.114:0/3703400057 learned_addr learned my addr 172.21.15.114:0/3703400057 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1205700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97bc0b3370 0x7f97bc0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c0a04700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 0x7f97bc0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 -- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 msgr2=0x7f97bc0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 0x7f97bc0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 -- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97bc0b3370 msgr2=0x7f97bc0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97bc0b3370 0x7f97bc0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 -- 172.21.15.114:0/3703400057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b4009b30 con 0x7f97bc15bdd0 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c0a04700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 0x7f97bc0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c1a06700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0716d0 tx=0x7f97b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40048b0 con 0x7f97bc15bdd0 2023-11-24T06:27:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b40099b0 con 0x7f97bc15bdd0 2023-11-24T06:27:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b401c680 con 0x7f97bc15bdd0 2023-11-24T06:27:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc0ccb00 con 0x7f97bc15bdd0 2023-11-24T06:27:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.626+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc0b4e10 con 0x7f97bc15bdd0 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.627+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc0cc680 con 0x7f97bc15bdd0 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.628+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f97b401c7e0 con 0x7f97bc15bdd0 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.628+0000 7f97b1ffb700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 0x7f97a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.628+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97b408b4f0 con 0x7f97bc15bdd0 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.628+0000 7f97c1205700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 0x7f97a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.628+0000 7f97c1205700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 0x7f97a006d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f97ac007fd0 tx=0x7f97ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.632+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97bc0cc680 con 0x7f97bc15bdd0 2023-11-24T06:27:50.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.820+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97bc166cf0 con 0x7f97a006b080 2023-11-24T06:27:50.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.827+0000 7f97b1ffb700 1 -- 172.21.15.114:0/3703400057 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286422 (secure 0 0 0) 0x7f97ac001cc0 con 0x7f97a006b080 2023-11-24T06:27:50.834 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 msgr2=0x7f97a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 0x7f97a006d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f97ac007fd0 tx=0x7f97ac009040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 msgr2=0x7f97bc150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0716d0 tx=0x7f97b4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 shutdown_connections 2023-11-24T06:27:50.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f97a006b080 0x7f97a006d540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97bc15bdd0 0x7f97bc150160 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97bc0b3370 0x7f97bc0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 --2- 172.21.15.114:0/3703400057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97bc0d2380 0x7f97bc0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 >> 172.21.15.114:0/3703400057 conn(0x7f97bc0731a0 msgr2=0x7f97bc1570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.830+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 shutdown_connections 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:50.831+0000 7f97c3469700 1 -- 172.21.15.114:0/3703400057 wait complete. 2023-11-24T06:27:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:27:50.846 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-24T06:27:49.753268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222684,"kb_used_data":6500,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752036,"statfs":{"total":773094113280,"available":772866084864,"internally_reserved":0,"allocated":6656000,"data_stored":3429776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001141"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188600+0000","last_change":"2023-11-24T06:27:45.196764+0000","last_active":"2023-11-24T06:27:46.188600+0000","last_peered":"2023-11-24T06:27:46.188600+0000","last_clean":"2023-11-24T06:27:46.188600+0000","last_became_active":"2023-11-24T06:27:45.196459+0000","last_became_peered":"2023-11-24T06:27:45.196459+0000","last_unstale":"2023-11-24T06:27:46.188600+0000","last_undegraded":"2023-11-24T06:27:46.188600+0000","last_fullsized":"2023-11-24T06:27:46.188600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:10.953646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439357+0000","last_change":"2023-11-24T06:27:45.199441+0000","last_active":"2023-11-24T06:27:46.439357+0000","last_peered":"2023-11-24T06:27:46.439357+0000","last_clean":"2023-11-24T06:27:46.439357+0000","last_became_active":"2023-11-24T06:27:45.199159+0000","last_became_peered":"2023-11-24T06:27:45.199159+0000","last_unstale":"2023-11-24T06:27:46.439357+0000","last_undegraded":"2023-11-24T06:27:46.439357+0000","last_fullsized":"2023-11-24T06:27:46.439357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:47.888236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876797+0000","last_change":"2023-11-24T06:27:43.876797+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876797+0000","last_undegraded":"2023-11-24T06:27:43.876797+0000","last_fullsized":"2023-11-24T06:27:43.876797+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.880382+0000","last_change":"2023-11-24T06:27:43.880382+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.880382+0000","last_undegraded":"2023-11-24T06:27:43.880382+0000","last_fullsized":"2023-11-24T06:27:43.880382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188111+0000","last_change":"2023-11-24T06:27:45.197360+0000","last_active":"2023-11-24T06:27:46.188111+0000","last_peered":"2023-11-24T06:27:46.188111+0000","last_clean":"2023-11-24T06:27:46.188111+0000","last_became_active":"2023-11-24T06:27:45.197045+0000","last_became_peered":"2023-11-24T06:27:45.197045+0000","last_unstale":"2023-11-24T06:27:46.188111+0000","last_undegraded":"2023-11-24T06:27:46.188111+0000","last_fullsized":"2023-11-24T06:27:46.188111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:50.810177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501275+0000","last_change":"2023-11-24T06:27:45.187750+0000","last_active":"2023-11-24T06:27:46.501275+0000","last_peered":"2023-11-24T06:27:46.501275+0000","last_clean":"2023-11-24T06:27:46.501275+0000","last_became_active":"2023-11-24T06:27:45.187445+0000","last_became_peered":"2023-11-24T06:27:45.187445+0000","last_unstale":"2023-11-24T06:27:46.501275+0000","last_undegraded":"2023-11-24T06:27:46.501275+0000","last_fullsized":"2023-11-24T06:27:46.501275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:06.517029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.884229+0000","last_change":"2023-11-24T06:27:43.884229+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.884229+0000","last_undegraded":"2023-11-24T06:27:43.884229+0000","last_fullsized":"2023-11-24T06:27:43.884229+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439075+0000","last_change":"2023-11-24T06:27:45.193543+0000","last_active":"2023-11-24T06:27:46.439075+0000","last_peered":"2023-11-24T06:27:46.439075+0000","last_clean":"2023-11-24T06:27:46.439075+0000","last_became_active":"2023-11-24T06:27:45.192904+0000","last_became_peered":"2023-11-24T06:27:45.192904+0000","last_unstale":"2023-11-24T06:27:46.439075+0000","last_undegraded":"2023-11-24T06:27:46.439075+0000","last_fullsized":"2023-11-24T06:27:46.439075+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:47.980944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881743+0000","last_change":"2023-11-24T06:27:43.881743+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881743+0000","last_undegraded":"2023-11-24T06:27:43.881743+0000","last_fullsized":"2023-11-24T06:27:43.881743+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875949+0000","last_change":"2023-11-24T06:27:43.875949+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875949+0000","last_undegraded":"2023-11-24T06:27:43.875949+0000","last_fullsized":"2023-11-24T06:27:43.875949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876187+0000","last_change":"2023-11-24T06:27:43.876187+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876187+0000","last_undegraded":"2023-11-24T06:27:43.876187+0000","last_fullsized":"2023-11-24T06:27:43.876187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878485+0000","last_change":"2023-11-24T06:27:43.878485+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878485+0000","last_undegraded":"2023-11-24T06:27:43.878485+0000","last_fullsized":"2023-11-24T06:27:43.878485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874177+0000","last_change":"2023-11-24T06:27:43.874177+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874177+0000","last_undegraded":"2023-11-24T06:27:43.874177+0000","last_fullsized":"2023-11-24T06:27:43.874177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.882087+0000","last_change":"2023-11-24T06:27:43.882087+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.882087+0000","last_undegraded":"2023-11-24T06:27:43.882087+0000","last_fullsized":"2023-11-24T06:27:43.882087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.883840+0000","last_change":"2023-11-24T06:27:43.883840+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.883840+0000","last_undegraded":"2023-11-24T06:27:43.883840+0000","last_fullsized":"2023-11-24T06:27:43.883840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501084+0000","last_change":"2023-11-24T06:27:45.192899+0000","last_active":"2023-11-24T06:27:46.501084+0000","last_peered":"2023-11-24T06:27:46.501084+0000","last_clean":"2023-11-24T06:27:46.501084+0000","last_became_active":"2023-11-24T06:27:45.192662+0000","last_became_peered":"2023-11-24T06:27:45.192662+0000","last_unstale":"2023-11-24T06:27:46.501084+0000","last_undegraded":"2023-11-24T06:27:46.501084+0000","last_fullsized":"2023-11-24T06:27:46.501084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:28.618271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439465+0000","last_change":"2023-11-24T06:27:45.193370+0000","last_active":"2023-11-24T06:27:46.439465+0000","last_peered":"2023-11-24T06:27:46.439465+0000","last_clean":"2023-11-24T06:27:46.439465+0000","last_became_active":"2023-11-24T06:27:45.193080+0000","last_became_peered":"2023-11-24T06:27:45.193080+0000","last_unstale":"2023-11-24T06:27:46.439465+0000","last_undegraded":"2023-11-24T06:27:46.439465+0000","last_fullsized":"2023-11-24T06:27:46.439465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:46.645009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875034+0000","last_change":"2023-11-24T06:27:43.875034+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875034+0000","last_undegraded":"2023-11-24T06:27:43.875034+0000","last_fullsized":"2023-11-24T06:27:43.875034+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188370+0000","last_change":"2023-11-24T06:27:45.196827+0000","last_active":"2023-11-24T06:27:46.188370+0000","last_peered":"2023-11-24T06:27:46.188370+0000","last_clean":"2023-11-24T06:27:46.188370+0000","last_became_active":"2023-11-24T06:27:45.196474+0000","last_became_peered":"2023-11-24T06:27:45.196474+0000","last_unstale":"2023-11-24T06:27:46.188370+0000","last_undegraded":"2023-11-24T06:27:46.188370+0000","last_fullsized":"2023-11-24T06:27:46.188370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:36.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439320+0000","last_change":"2023-11-24T06:27:45.199695+0000","last_active":"2023-11-24T06:27:46.439320+0000","last_peered":"2023-11-24T06:27:46.439320+0000","last_clean":"2023-11-24T06:27:46.439320+0000","last_became_active":"2023-11-24T06:27:45.199046+0000","last_became_peered":"2023-11-24T06:27:45.199046+0000","last_unstale":"2023-11-24T06:27:46.439320+0000","last_undegraded":"2023-11-24T06:27:46.439320+0000","last_fullsized":"2023-11-24T06:27:46.439320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:02.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875751+0000","last_change":"2023-11-24T06:27:43.875751+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875751+0000","last_undegraded":"2023-11-24T06:27:43.875751+0000","last_fullsized":"2023-11-24T06:27:43.875751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439113+0000","last_change":"2023-11-24T06:27:45.196449+0000","last_active":"2023-11-24T06:27:46.439113+0000","last_peered":"2023-11-24T06:27:46.439113+0000","last_clean":"2023-11-24T06:27:46.439113+0000","last_became_active":"2023-11-24T06:27:45.196184+0000","last_became_peered":"2023-11-24T06:27:45.196184+0000","last_unstale":"2023-11-24T06:27:46.439113+0000","last_undegraded":"2023-11-24T06:27:46.439113+0000","last_fullsized":"2023-11-24T06:27:46.439113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:14.864903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873638+0000","last_change":"2023-11-24T06:27:43.873638+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873638+0000","last_undegraded":"2023-11-24T06:27:43.873638+0000","last_fullsized":"2023-11-24T06:27:43.873638+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875332+0000","last_change":"2023-11-24T06:27:43.875332+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875332+0000","last_undegraded":"2023-11-24T06:27:43.875332+0000","last_fullsized":"2023-11-24T06:27:43.875332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188504+0000","last_change":"2023-11-24T06:27:45.199317+0000","last_active":"2023-11-24T06:27:46.188504+0000","last_peered":"2023-11-24T06:27:46.188504+0000","last_clean":"2023-11-24T06:27:46.188504+0000","last_became_active":"2023-11-24T06:27:45.198954+0000","last_became_peered":"2023-11-24T06:27:45.198954+0000","last_unstale":"2023-11-24T06:27:46.188504+0000","last_undegraded":"2023-11-24T06:27:46.188504+0000","last_fullsized":"2023-11-24T06:27:46.188504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:52.546554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878658+0000","last_change":"2023-11-24T06:27:43.878658+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878658+0000","last_undegraded":"2023-11-24T06:27:43.878658+0000","last_fullsized":"2023-11-24T06:27:43.878658+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501543+0000","last_change":"2023-11-24T06:27:45.191845+0000","last_active":"2023-11-24T06:27:46.501543+0000","last_peered":"2023-11-24T06:27:46.501543+0000","last_clean":"2023-11-24T06:27:46.501543+0000","last_became_active":"2023-11-24T06:27:45.191565+0000","last_became_peered":"2023-11-24T06:27:45.191565+0000","last_unstale":"2023-11-24T06:27:46.501543+0000","last_undegraded":"2023-11-24T06:27:46.501543+0000","last_fullsized":"2023-11-24T06:27:46.501543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:10.419872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188231+0000","last_change":"2023-11-24T06:27:45.191652+0000","last_active":"2023-11-24T06:27:46.188231+0000","last_peered":"2023-11-24T06:27:46.188231+0000","last_clean":"2023-11-24T06:27:46.188231+0000","last_became_active":"2023-11-24T06:27:45.191360+0000","last_became_peered":"2023-11-24T06:27:45.191360+0000","last_unstale":"2023-11-24T06:27:46.188231+0000","last_undegraded":"2023-11-24T06:27:46.188231+0000","last_fullsized":"2023-11-24T06:27:46.188231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:17.745755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.191429+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.191159+0000","last_became_peered":"2023-11-24T06:27:45.191159+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.196091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874041+0000","last_change":"2023-11-24T06:27:43.874041+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874041+0000","last_undegraded":"2023-11-24T06:27:43.874041+0000","last_fullsized":"2023-11-24T06:27:43.874041+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.187889+0000","last_change":"2023-11-24T06:27:45.199361+0000","last_active":"2023-11-24T06:27:46.187889+0000","last_peered":"2023-11-24T06:27:46.187889+0000","last_clean":"2023-11-24T06:27:46.187889+0000","last_became_active":"2023-11-24T06:27:45.199093+0000","last_became_peered":"2023-11-24T06:27:45.199093+0000","last_unstale":"2023-11-24T06:27:46.187889+0000","last_undegraded":"2023-11-24T06:27:46.187889+0000","last_fullsized":"2023-11-24T06:27:46.187889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:20.032382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501160+0000","last_change":"2023-11-24T06:27:45.194242+0000","last_active":"2023-11-24T06:27:46.501160+0000","last_peered":"2023-11-24T06:27:46.501160+0000","last_clean":"2023-11-24T06:27:46.501160+0000","last_became_active":"2023-11-24T06:27:45.193957+0000","last_became_peered":"2023-11-24T06:27:45.193957+0000","last_unstale":"2023-11-24T06:27:46.501160+0000","last_undegraded":"2023-11-24T06:27:46.501160+0000","last_fullsized":"2023-11-24T06:27:46.501160+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.755035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501812+0000","last_change":"2023-11-24T06:27:45.187022+0000","last_active":"2023-11-24T06:27:46.501812+0000","last_peered":"2023-11-24T06:27:46.501812+0000","last_clean":"2023-11-24T06:27:46.501812+0000","last_became_active":"2023-11-24T06:27:45.186746+0000","last_became_peered":"2023-11-24T06:27:45.186746+0000","last_unstale":"2023-11-24T06:27:46.501812+0000","last_undegraded":"2023-11-24T06:27:46.501812+0000","last_fullsized":"2023-11-24T06:27:46.501812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:07.653354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212303+0000","last_change":"2023-11-24T06:27:43.887983+0000","last_active":"2023-11-24T06:27:45.212303+0000","last_peered":"2023-11-24T06:27:45.212303+0000","last_clean":"2023-11-24T06:27:45.212303+0000","last_became_active":"2023-11-24T06:27:43.887687+0000","last_became_peered":"2023-11-24T06:27:43.887687+0000","last_unstale":"2023-11-24T06:27:45.212303+0000","last_undegraded":"2023-11-24T06:27:45.212303+0000","last_fullsized":"2023-11-24T06:27:45.212303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.876827+0000","last_change":"2023-11-24T06:27:43.876827+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.876827+0000","last_undegraded":"2023-11-24T06:27:43.876827+0000","last_fullsized":"2023-11-24T06:27:43.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207909+0000","last_change":"2023-11-24T06:27:43.887478+0000","last_active":"2023-11-24T06:27:45.207909+0000","last_peered":"2023-11-24T06:27:45.207909+0000","last_clean":"2023-11-24T06:27:45.207909+0000","last_became_active":"2023-11-24T06:27:43.886904+0000","last_became_peered":"2023-11-24T06:27:43.886904+0000","last_unstale":"2023-11-24T06:27:45.207909+0000","last_undegraded":"2023-11-24T06:27:45.207909+0000","last_fullsized":"2023-11-24T06:27:45.207909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:57.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501874+0000","last_change":"2023-11-24T06:27:45.184656+0000","last_active":"2023-11-24T06:27:46.501874+0000","last_peered":"2023-11-24T06:27:46.501874+0000","last_clean":"2023-11-24T06:27:46.501874+0000","last_became_active":"2023-11-24T06:27:45.184372+0000","last_became_peered":"2023-11-24T06:27:45.184372+0000","last_unstale":"2023-11-24T06:27:46.501874+0000","last_undegraded":"2023-11-24T06:27:46.501874+0000","last_fullsized":"2023-11-24T06:27:46.501874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:43.478543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209895+0000","last_change":"2023-11-24T06:27:43.893948+0000","last_active":"2023-11-24T06:27:45.209895+0000","last_peered":"2023-11-24T06:27:45.209895+0000","last_clean":"2023-11-24T06:27:45.209895+0000","last_became_active":"2023-11-24T06:27:43.893661+0000","last_became_peered":"2023-11-24T06:27:43.893661+0000","last_unstale":"2023-11-24T06:27:45.209895+0000","last_undegraded":"2023-11-24T06:27:45.209895+0000","last_fullsized":"2023-11-24T06:27:45.209895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:38:51.432906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501588+0000","last_change":"2023-11-24T06:27:45.189667+0000","last_active":"2023-11-24T06:27:46.501588+0000","last_peered":"2023-11-24T06:27:46.501588+0000","last_clean":"2023-11-24T06:27:46.501588+0000","last_became_active":"2023-11-24T06:27:45.189394+0000","last_became_peered":"2023-11-24T06:27:45.189394+0000","last_unstale":"2023-11-24T06:27:46.501588+0000","last_undegraded":"2023-11-24T06:27:46.501588+0000","last_fullsized":"2023-11-24T06:27:46.501588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:18.377171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887326+0000","last_change":"2023-11-24T06:27:43.884164+0000","last_active":"2023-11-24T06:27:43.887326+0000","last_peered":"2023-11-24T06:27:43.887326+0000","last_clean":"2023-11-24T06:27:43.887326+0000","last_became_active":"2023-11-24T06:27:43.883886+0000","last_became_peered":"2023-11-24T06:27:43.883886+0000","last_unstale":"2023-11-24T06:27:43.887326+0000","last_undegraded":"2023-11-24T06:27:43.887326+0000","last_fullsized":"2023-11-24T06:27:43.887326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:35.939429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439144+0000","last_change":"2023-11-24T06:27:45.196404+0000","last_active":"2023-11-24T06:27:46.439144+0000","last_peered":"2023-11-24T06:27:46.439144+0000","last_clean":"2023-11-24T06:27:46.439144+0000","last_became_active":"2023-11-24T06:27:45.196180+0000","last_became_peered":"2023-11-24T06:27:45.196180+0000","last_unstale":"2023-11-24T06:27:46.439144+0000","last_undegraded":"2023-11-24T06:27:46.439144+0000","last_fullsized":"2023-11-24T06:27:46.439144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:01.939068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891238+0000","last_change":"2023-11-24T06:27:43.887860+0000","last_active":"2023-11-24T06:27:43.891238+0000","last_peered":"2023-11-24T06:27:43.891238+0000","last_clean":"2023-11-24T06:27:43.891238+0000","last_became_active":"2023-11-24T06:27:43.887522+0000","last_became_peered":"2023-11-24T06:27:43.887522+0000","last_unstale":"2023-11-24T06:27:43.891238+0000","last_undegraded":"2023-11-24T06:27:43.891238+0000","last_fullsized":"2023-11-24T06:27:43.891238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:18.289207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439483+0000","last_change":"2023-11-24T06:27:45.190694+0000","last_active":"2023-11-24T06:27:46.439483+0000","last_peered":"2023-11-24T06:27:46.439483+0000","last_clean":"2023-11-24T06:27:46.439483+0000","last_became_active":"2023-11-24T06:27:45.190410+0000","last_became_peered":"2023-11-24T06:27:45.190410+0000","last_unstale":"2023-11-24T06:27:46.439483+0000","last_undegraded":"2023-11-24T06:27:46.439483+0000","last_fullsized":"2023-11-24T06:27:46.439483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:55.059669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208021+0000","last_change":"2023-11-24T06:27:43.888671+0000","last_active":"2023-11-24T06:27:45.208021+0000","last_peered":"2023-11-24T06:27:45.208021+0000","last_clean":"2023-11-24T06:27:45.208021+0000","last_became_active":"2023-11-24T06:27:43.888398+0000","last_became_peered":"2023-11-24T06:27:43.888398+0000","last_unstale":"2023-11-24T06:27:45.208021+0000","last_undegraded":"2023-11-24T06:27:45.208021+0000","last_fullsized":"2023-11-24T06:27:45.208021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:04.828359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873693+0000","last_change":"2023-11-24T06:27:43.873693+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873693+0000","last_undegraded":"2023-11-24T06:27:43.873693+0000","last_fullsized":"2023-11-24T06:27:43.873693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893351+0000","last_change":"2023-11-24T06:27:43.888597+0000","last_active":"2023-11-24T06:27:43.893351+0000","last_peered":"2023-11-24T06:27:43.893351+0000","last_clean":"2023-11-24T06:27:43.893351+0000","last_became_active":"2023-11-24T06:27:43.888055+0000","last_became_peered":"2023-11-24T06:27:43.888055+0000","last_unstale":"2023-11-24T06:27:43.893351+0000","last_undegraded":"2023-11-24T06:27:43.893351+0000","last_fullsized":"2023-11-24T06:27:43.893351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439166+0000","last_change":"2023-11-24T06:27:45.194260+0000","last_active":"2023-11-24T06:27:46.439166+0000","last_peered":"2023-11-24T06:27:46.439166+0000","last_clean":"2023-11-24T06:27:46.439166+0000","last_became_active":"2023-11-24T06:27:45.193685+0000","last_became_peered":"2023-11-24T06:27:45.193685+0000","last_unstale":"2023-11-24T06:27:46.439166+0000","last_undegraded":"2023-11-24T06:27:46.439166+0000","last_fullsized":"2023-11-24T06:27:46.439166+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:56.547263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893456+0000","last_change":"2023-11-24T06:27:43.890543+0000","last_active":"2023-11-24T06:27:43.893456+0000","last_peered":"2023-11-24T06:27:43.893456+0000","last_clean":"2023-11-24T06:27:43.893456+0000","last_became_active":"2023-11-24T06:27:43.890213+0000","last_became_peered":"2023-11-24T06:27:43.890213+0000","last_unstale":"2023-11-24T06:27:43.893456+0000","last_undegraded":"2023-11-24T06:27:43.893456+0000","last_fullsized":"2023-11-24T06:27:43.893456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:21.791667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881850+0000","last_change":"2023-11-24T06:27:43.881850+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881850+0000","last_undegraded":"2023-11-24T06:27:43.881850+0000","last_fullsized":"2023-11-24T06:27:43.881850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893344+0000","last_change":"2023-11-24T06:27:43.888862+0000","last_active":"2023-11-24T06:27:43.893344+0000","last_peered":"2023-11-24T06:27:43.893344+0000","last_clean":"2023-11-24T06:27:43.893344+0000","last_became_active":"2023-11-24T06:27:43.888560+0000","last_became_peered":"2023-11-24T06:27:43.888560+0000","last_unstale":"2023-11-24T06:27:43.893344+0000","last_undegraded":"2023-11-24T06:27:43.893344+0000","last_fullsized":"2023-11-24T06:27:43.893344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:25.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.873668+0000","last_change":"2023-11-24T06:27:43.873668+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.873668+0000","last_undegraded":"2023-11-24T06:27:43.873668+0000","last_fullsized":"2023-11-24T06:27:43.873668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.187762+0000","last_change":"2023-11-24T06:27:43.886888+0000","last_active":"2023-11-24T06:27:45.187762+0000","last_peered":"2023-11-24T06:27:45.187762+0000","last_clean":"2023-11-24T06:27:45.187762+0000","last_became_active":"2023-11-24T06:27:43.886584+0000","last_became_peered":"2023-11-24T06:27:43.886584+0000","last_unstale":"2023-11-24T06:27:45.187762+0000","last_undegraded":"2023-11-24T06:27:45.187762+0000","last_fullsized":"2023-11-24T06:27:45.187762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:17:04.535818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878276+0000","last_change":"2023-11-24T06:27:43.878276+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878276+0000","last_undegraded":"2023-11-24T06:27:43.878276+0000","last_fullsized":"2023-11-24T06:27:43.878276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891262+0000","last_change":"2023-11-24T06:27:43.887129+0000","last_active":"2023-11-24T06:27:43.891262+0000","last_peered":"2023-11-24T06:27:43.891262+0000","last_clean":"2023-11-24T06:27:43.891262+0000","last_became_active":"2023-11-24T06:27:43.886873+0000","last_became_peered":"2023-11-24T06:27:43.886873+0000","last_unstale":"2023-11-24T06:27:43.891262+0000","last_undegraded":"2023-11-24T06:27:43.891262+0000","last_fullsized":"2023-11-24T06:27:43.891262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875846+0000","last_change":"2023-11-24T06:27:43.875846+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875846+0000","last_undegraded":"2023-11-24T06:27:43.875846+0000","last_fullsized":"2023-11-24T06:27:43.875846+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893615+0000","last_change":"2023-11-24T06:27:43.891037+0000","last_active":"2023-11-24T06:27:43.893615+0000","last_peered":"2023-11-24T06:27:43.893615+0000","last_clean":"2023-11-24T06:27:43.893615+0000","last_became_active":"2023-11-24T06:27:43.890756+0000","last_became_peered":"2023-11-24T06:27:43.890756+0000","last_unstale":"2023-11-24T06:27:43.893615+0000","last_undegraded":"2023-11-24T06:27:43.893615+0000","last_fullsized":"2023-11-24T06:27:43.893615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:56.949114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877698+0000","last_change":"2023-11-24T06:27:43.877698+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877698+0000","last_undegraded":"2023-11-24T06:27:43.877698+0000","last_fullsized":"2023-11-24T06:27:43.877698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209609+0000","last_change":"2023-11-24T06:27:43.887228+0000","last_active":"2023-11-24T06:27:45.209609+0000","last_peered":"2023-11-24T06:27:45.209609+0000","last_clean":"2023-11-24T06:27:45.209609+0000","last_became_active":"2023-11-24T06:27:43.886820+0000","last_became_peered":"2023-11-24T06:27:43.886820+0000","last_unstale":"2023-11-24T06:27:45.209609+0000","last_undegraded":"2023-11-24T06:27:45.209609+0000","last_fullsized":"2023-11-24T06:27:45.209609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:47.443503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875335+0000","last_change":"2023-11-24T06:27:43.875335+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875335+0000","last_undegraded":"2023-11-24T06:27:43.875335+0000","last_fullsized":"2023-11-24T06:27:43.875335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.881677+0000","last_change":"2023-11-24T06:27:43.880061+0000","last_active":"2023-11-24T06:27:43.881677+0000","last_peered":"2023-11-24T06:27:43.881677+0000","last_clean":"2023-11-24T06:27:43.881677+0000","last_became_active":"2023-11-24T06:27:43.879790+0000","last_became_peered":"2023-11-24T06:27:43.879790+0000","last_unstale":"2023-11-24T06:27:43.881677+0000","last_undegraded":"2023-11-24T06:27:43.881677+0000","last_fullsized":"2023-11-24T06:27:43.881677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:16.744160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.881584+0000","last_change":"2023-11-24T06:27:43.881584+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.881584+0000","last_undegraded":"2023-11-24T06:27:43.881584+0000","last_fullsized":"2023-11-24T06:27:43.881584+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885337+0000","last_change":"2023-11-24T06:27:43.881814+0000","last_active":"2023-11-24T06:27:43.885337+0000","last_peered":"2023-11-24T06:27:43.885337+0000","last_clean":"2023-11-24T06:27:43.885337+0000","last_became_active":"2023-11-24T06:27:43.881477+0000","last_became_peered":"2023-11-24T06:27:43.881477+0000","last_unstale":"2023-11-24T06:27:43.885337+0000","last_undegraded":"2023-11-24T06:27:43.885337+0000","last_fullsized":"2023-11-24T06:27:43.885337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874261+0000","last_change":"2023-11-24T06:27:43.874261+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874261+0000","last_undegraded":"2023-11-24T06:27:43.874261+0000","last_fullsized":"2023-11-24T06:27:43.874261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.888453+0000","last_change":"2023-11-24T06:27:43.885895+0000","last_active":"2023-11-24T06:27:43.888453+0000","last_peered":"2023-11-24T06:27:43.888453+0000","last_clean":"2023-11-24T06:27:43.888453+0000","last_became_active":"2023-11-24T06:27:43.885623+0000","last_became_peered":"2023-11-24T06:27:43.885623+0000","last_unstale":"2023-11-24T06:27:43.888453+0000","last_undegraded":"2023-11-24T06:27:43.888453+0000","last_fullsized":"2023-11-24T06:27:43.888453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:50.505178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874451+0000","last_change":"2023-11-24T06:27:43.874451+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874451+0000","last_undegraded":"2023-11-24T06:27:43.874451+0000","last_fullsized":"2023-11-24T06:27:43.874451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209513+0000","last_change":"2023-11-24T06:27:43.887376+0000","last_active":"2023-11-24T06:27:45.209513+0000","last_peered":"2023-11-24T06:27:45.209513+0000","last_clean":"2023-11-24T06:27:45.209513+0000","last_became_active":"2023-11-24T06:27:43.887125+0000","last_became_peered":"2023-11-24T06:27:43.887125+0000","last_unstale":"2023-11-24T06:27:45.209513+0000","last_undegraded":"2023-11-24T06:27:45.209513+0000","last_fullsized":"2023-11-24T06:27:45.209513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:06.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.878320+0000","last_change":"2023-11-24T06:27:43.878320+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.878320+0000","last_undegraded":"2023-11-24T06:27:43.878320+0000","last_fullsized":"2023-11-24T06:27:43.878320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208265+0000","last_change":"2023-11-24T06:27:43.887382+0000","last_active":"2023-11-24T06:27:45.208265+0000","last_peered":"2023-11-24T06:27:45.208265+0000","last_clean":"2023-11-24T06:27:45.208265+0000","last_became_active":"2023-11-24T06:27:43.887150+0000","last_became_peered":"2023-11-24T06:27:43.887150+0000","last_unstale":"2023-11-24T06:27:45.208265+0000","last_undegraded":"2023-11-24T06:27:45.208265+0000","last_fullsized":"2023-11-24T06:27:45.208265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.841385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.197917+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.197646+0000","last_became_peered":"2023-11-24T06:27:45.197646+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:12.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887186+0000","last_change":"2023-11-24T06:27:43.884248+0000","last_active":"2023-11-24T06:27:43.887186+0000","last_peered":"2023-11-24T06:27:43.887186+0000","last_clean":"2023-11-24T06:27:43.887186+0000","last_became_active":"2023-11-24T06:27:43.883575+0000","last_became_peered":"2023-11-24T06:27:43.883575+0000","last_unstale":"2023-11-24T06:27:43.887186+0000","last_undegraded":"2023-11-24T06:27:43.887186+0000","last_fullsized":"2023-11-24T06:27:43.887186+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:17:46.473315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874503+0000","last_change":"2023-11-24T06:27:43.874503+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874503+0000","last_undegraded":"2023-11-24T06:27:43.874503+0000","last_fullsized":"2023-11-24T06:27:43.874503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179085+0000","last_change":"2023-11-24T06:27:43.890800+0000","last_active":"2023-11-24T06:27:45.179085+0000","last_peered":"2023-11-24T06:27:45.179085+0000","last_clean":"2023-11-24T06:27:45.179085+0000","last_became_active":"2023-11-24T06:27:43.890508+0000","last_became_peered":"2023-11-24T06:27:43.890508+0000","last_unstale":"2023-11-24T06:27:45.179085+0000","last_undegraded":"2023-11-24T06:27:45.179085+0000","last_fullsized":"2023-11-24T06:27:45.179085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:04.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877020+0000","last_change":"2023-11-24T06:27:43.877020+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877020+0000","last_undegraded":"2023-11-24T06:27:43.877020+0000","last_fullsized":"2023-11-24T06:27:43.877020+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212410+0000","last_change":"2023-11-24T06:27:43.888453+0000","last_active":"2023-11-24T06:27:45.212410+0000","last_peered":"2023-11-24T06:27:45.212410+0000","last_clean":"2023-11-24T06:27:45.212410+0000","last_became_active":"2023-11-24T06:27:43.888169+0000","last_became_peered":"2023-11-24T06:27:43.888169+0000","last_unstale":"2023-11-24T06:27:45.212410+0000","last_undegraded":"2023-11-24T06:27:45.212410+0000","last_fullsized":"2023-11-24T06:27:45.212410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:37.661869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501526+0000","last_change":"2023-11-24T06:27:45.192208+0000","last_active":"2023-11-24T06:27:46.501526+0000","last_peered":"2023-11-24T06:27:46.501526+0000","last_clean":"2023-11-24T06:27:46.501526+0000","last_became_active":"2023-11-24T06:27:45.191906+0000","last_became_peered":"2023-11-24T06:27:45.191906+0000","last_unstale":"2023-11-24T06:27:46.501526+0000","last_undegraded":"2023-11-24T06:27:46.501526+0000","last_fullsized":"2023-11-24T06:27:46.501526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:28.930496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893367+0000","last_change":"2023-11-24T06:27:43.886183+0000","last_active":"2023-11-24T06:27:43.893367+0000","last_peered":"2023-11-24T06:27:43.893367+0000","last_clean":"2023-11-24T06:27:43.893367+0000","last_became_active":"2023-11-24T06:27:43.885892+0000","last_became_peered":"2023-11-24T06:27:43.885892+0000","last_unstale":"2023-11-24T06:27:43.893367+0000","last_undegraded":"2023-11-24T06:27:43.893367+0000","last_fullsized":"2023-11-24T06:27:43.893367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:09.952050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501311+0000","last_change":"2023-11-24T06:27:45.187149+0000","last_active":"2023-11-24T06:27:46.501311+0000","last_peered":"2023-11-24T06:27:46.501311+0000","last_clean":"2023-11-24T06:27:46.501311+0000","last_became_active":"2023-11-24T06:27:45.186822+0000","last_became_peered":"2023-11-24T06:27:45.186822+0000","last_unstale":"2023-11-24T06:27:46.501311+0000","last_undegraded":"2023-11-24T06:27:46.501311+0000","last_fullsized":"2023-11-24T06:27:46.501311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:56.389229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893378+0000","last_change":"2023-11-24T06:27:43.890788+0000","last_active":"2023-11-24T06:27:43.893378+0000","last_peered":"2023-11-24T06:27:43.893378+0000","last_clean":"2023-11-24T06:27:43.893378+0000","last_became_active":"2023-11-24T06:27:43.890453+0000","last_became_peered":"2023-11-24T06:27:43.890453+0000","last_unstale":"2023-11-24T06:27:43.893378+0000","last_undegraded":"2023-11-24T06:27:43.893378+0000","last_fullsized":"2023-11-24T06:27:43.893378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877120+0000","last_change":"2023-11-24T06:27:43.877120+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877120+0000","last_undegraded":"2023-11-24T06:27:43.877120+0000","last_fullsized":"2023-11-24T06:27:43.877120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893473+0000","last_change":"2023-11-24T06:27:43.889734+0000","last_active":"2023-11-24T06:27:43.893473+0000","last_peered":"2023-11-24T06:27:43.893473+0000","last_clean":"2023-11-24T06:27:43.893473+0000","last_became_active":"2023-11-24T06:27:43.889470+0000","last_became_peered":"2023-11-24T06:27:43.889470+0000","last_unstale":"2023-11-24T06:27:43.893473+0000","last_undegraded":"2023-11-24T06:27:43.893473+0000","last_fullsized":"2023-11-24T06:27:43.893473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:45.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.877591+0000","last_change":"2023-11-24T06:27:43.877591+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.877591+0000","last_undegraded":"2023-11-24T06:27:43.877591+0000","last_fullsized":"2023-11-24T06:27:43.877591+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501897+0000","last_change":"2023-11-24T06:27:43.889511+0000","last_active":"2023-11-24T06:27:46.501897+0000","last_peered":"2023-11-24T06:27:46.501897+0000","last_clean":"2023-11-24T06:27:46.501897+0000","last_became_active":"2023-11-24T06:27:43.889234+0000","last_became_peered":"2023-11-24T06:27:43.889234+0000","last_unstale":"2023-11-24T06:27:46.501897+0000","last_undegraded":"2023-11-24T06:27:46.501897+0000","last_fullsized":"2023-11-24T06:27:46.501897+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:01.413877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875653+0000","last_change":"2023-11-24T06:27:43.875653+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875653+0000","last_undegraded":"2023-11-24T06:27:43.875653+0000","last_fullsized":"2023-11-24T06:27:43.875653+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.887987+0000","last_change":"2023-11-24T06:27:43.886417+0000","last_active":"2023-11-24T06:27:43.887987+0000","last_peered":"2023-11-24T06:27:43.887987+0000","last_clean":"2023-11-24T06:27:43.887987+0000","last_became_active":"2023-11-24T06:27:43.886110+0000","last_became_peered":"2023-11-24T06:27:43.886110+0000","last_unstale":"2023-11-24T06:27:43.887987+0000","last_undegraded":"2023-11-24T06:27:43.887987+0000","last_fullsized":"2023-11-24T06:27:43.887987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:35.558430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501695+0000","last_change":"2023-11-24T06:27:45.184597+0000","last_active":"2023-11-24T06:27:46.501695+0000","last_peered":"2023-11-24T06:27:46.501695+0000","last_clean":"2023-11-24T06:27:46.501695+0000","last_became_active":"2023-11-24T06:27:45.184299+0000","last_became_peered":"2023-11-24T06:27:45.184299+0000","last_unstale":"2023-11-24T06:27:46.501695+0000","last_undegraded":"2023-11-24T06:27:46.501695+0000","last_fullsized":"2023-11-24T06:27:46.501695+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:51.039635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.891002+0000","last_change":"2023-11-24T06:27:43.887202+0000","last_active":"2023-11-24T06:27:43.891002+0000","last_peered":"2023-11-24T06:27:43.891002+0000","last_clean":"2023-11-24T06:27:43.891002+0000","last_became_active":"2023-11-24T06:27:43.886926+0000","last_became_peered":"2023-11-24T06:27:43.886926+0000","last_unstale":"2023-11-24T06:27:43.891002+0000","last_undegraded":"2023-11-24T06:27:43.891002+0000","last_fullsized":"2023-11-24T06:27:43.891002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439423+0000","last_change":"2023-11-24T06:27:45.199563+0000","last_active":"2023-11-24T06:27:46.439423+0000","last_peered":"2023-11-24T06:27:46.439423+0000","last_clean":"2023-11-24T06:27:46.439423+0000","last_became_active":"2023-11-24T06:27:45.199267+0000","last_became_peered":"2023-11-24T06:27:45.199267+0000","last_unstale":"2023-11-24T06:27:46.439423+0000","last_undegraded":"2023-11-24T06:27:46.439423+0000","last_fullsized":"2023-11-24T06:27:46.439423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:45.747801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893321+0000","last_change":"2023-11-24T06:27:43.885639+0000","last_active":"2023-11-24T06:27:43.893321+0000","last_peered":"2023-11-24T06:27:43.893321+0000","last_clean":"2023-11-24T06:27:43.893321+0000","last_became_active":"2023-11-24T06:27:43.885321+0000","last_became_peered":"2023-11-24T06:27:43.885321+0000","last_unstale":"2023-11-24T06:27:43.893321+0000","last_undegraded":"2023-11-24T06:27:43.893321+0000","last_fullsized":"2023-11-24T06:27:43.893321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.874894+0000","last_change":"2023-11-24T06:27:43.874894+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.874894+0000","last_undegraded":"2023-11-24T06:27:43.874894+0000","last_fullsized":"2023-11-24T06:27:43.874894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893401+0000","last_change":"2023-11-24T06:27:43.888919+0000","last_active":"2023-11-24T06:27:43.893401+0000","last_peered":"2023-11-24T06:27:43.893401+0000","last_clean":"2023-11-24T06:27:43.893401+0000","last_became_active":"2023-11-24T06:27:43.888593+0000","last_became_peered":"2023-11-24T06:27:43.888593+0000","last_unstale":"2023-11-24T06:27:43.893401+0000","last_undegraded":"2023-11-24T06:27:43.893401+0000","last_fullsized":"2023-11-24T06:27:43.893401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:01.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-24T06:27:43.875691+0000","last_change":"2023-11-24T06:27:43.875691+0000","last_active":"2023-11-24T06:27:43.868182+0000","last_peered":"2023-11-24T06:27:43.868182+0000","last_clean":"2023-11-24T06:27:43.868182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:27:43.875691+0000","last_undegraded":"2023-11-24T06:27:43.875691+0000","last_fullsized":"2023-11-24T06:27:43.875691+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.893416+0000","last_change":"2023-11-24T06:27:43.889027+0000","last_active":"2023-11-24T06:27:43.893416+0000","last_peered":"2023-11-24T06:27:43.893416+0000","last_clean":"2023-11-24T06:27:43.893416+0000","last_became_active":"2023-11-24T06:27:43.888756+0000","last_became_peered":"2023-11-24T06:27:43.888756+0000","last_unstale":"2023-11-24T06:27:43.893416+0000","last_undegraded":"2023-11-24T06:27:43.893416+0000","last_fullsized":"2023-11-24T06:27:43.893416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:35:59.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439231+0000","last_change":"2023-11-24T06:27:45.194043+0000","last_active":"2023-11-24T06:27:46.439231+0000","last_peered":"2023-11-24T06:27:46.439231+0000","last_clean":"2023-11-24T06:27:46.439231+0000","last_became_active":"2023-11-24T06:27:45.192605+0000","last_became_peered":"2023-11-24T06:27:45.192605+0000","last_unstale":"2023-11-24T06:27:46.439231+0000","last_undegraded":"2023-11-24T06:27:46.439231+0000","last_fullsized":"2023-11-24T06:27:46.439231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:59.614993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.180294+0000","last_change":"2023-11-24T06:27:43.889915+0000","last_active":"2023-11-24T06:27:45.180294+0000","last_peered":"2023-11-24T06:27:45.180294+0000","last_clean":"2023-11-24T06:27:45.180294+0000","last_became_active":"2023-11-24T06:27:43.889576+0000","last_became_peered":"2023-11-24T06:27:43.889576+0000","last_unstale":"2023-11-24T06:27:45.180294+0000","last_undegraded":"2023-11-24T06:27:45.180294+0000","last_fullsized":"2023-11-24T06:27:45.180294+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:41:42.721820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439124+0000","last_change":"2023-11-24T06:27:45.193323+0000","last_active":"2023-11-24T06:27:46.439124+0000","last_peered":"2023-11-24T06:27:46.439124+0000","last_clean":"2023-11-24T06:27:46.439124+0000","last_became_active":"2023-11-24T06:27:45.192548+0000","last_became_peered":"2023-11-24T06:27:45.192548+0000","last_unstale":"2023-11-24T06:27:46.439124+0000","last_undegraded":"2023-11-24T06:27:46.439124+0000","last_fullsized":"2023-11-24T06:27:46.439124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:09.532953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T06:27:43.885063+0000","last_change":"2023-11-24T06:27:43.882518+0000","last_active":"2023-11-24T06:27:43.885063+0000","last_peered":"2023-11-24T06:27:43.885063+0000","last_clean":"2023-11-24T06:27:43.885063+0000","last_became_active":"2023-11-24T06:27:43.882224+0000","last_became_peered":"2023-11-24T06:27:43.882224+0000","last_unstale":"2023-11-24T06:27:43.885063+0000","last_undegraded":"2023-11-24T06:27:43.885063+0000","last_fullsized":"2023-11-24T06:27:43.885063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:51.544959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-24T06:27:42.872822+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:27:42.872822+0000","last_peered":"2023-11-24T06:27:42.872822+0000","last_clean":"2023-11-24T06:27:42.872822+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:27:42.872822+0000","last_undegraded":"2023-11-24T06:27:42.872822+0000","last_fullsized":"2023-11-24T06:27:42.872822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.628}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.77600000000000002,"15min":0.86599999999999999},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.87,"5min":1.373,"15min":1.373},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.411},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.345,"15min":1.345},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.80700000000000005,"5min":1.286,"15min":1.286},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46800000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":1.379,"15min":1.379},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.37},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.27000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.307}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.57799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.315,"15min":0.313},"min":{"1min":0.20899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.254},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.29699999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51200000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":1.028,"15min":1.028},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91600000000000004,"5min":1.01,"15min":1.01},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.0369999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":1,"15min":1},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.316},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.751},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.84099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.85799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":276722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.496}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.377},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.309}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:53.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.982+0000 7f80bcdc9700 1 Processor -- start 2023-11-24T06:27:53.986 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.982+0000 7f80bcdc9700 1 -- start start 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.983+0000 7f80bcdc9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.983+0000 7f80bcdc9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80b80b1a70 0x7f80b8164b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.983+0000 7f80bcdc9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b80cfd40 con 0x7f80b815ae20 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.983+0000 7f80bcdc9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80d0020 con 0x7f80b80b1a70 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.983+0000 7f80bcdc9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80affa0 con 0x7f80b815b200 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36784/0 (socket says 172.21.15.114:36784) 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b6d9d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80b80b1a70 0x7f80b8164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 -- 172.21.15.114:0/3954706944 learned_addr learned my addr 172.21.15.114:0/3954706944 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b559a700 1 -- 172.21.15.114:0/3954706944 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354908636 0 0) 0x7f80b80cfd40 con 0x7f80b815ae20 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b559a700 1 -- 172.21.15.114:0/3954706944 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a00035b0 con 0x7f80b815ae20 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 -- 172.21.15.114:0/3954706944 >> v2:172.21.15.172:3300/0 conn(0x7f80b80b1a70 msgr2=0x7f80b8164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 --2- 172.21.15.114:0/3954706944 >> v2:172.21.15.172:3300/0 conn(0x7f80b80b1a70 0x7f80b8164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 -- 172.21.15.114:0/3954706944 >> v1:172.21.15.172:6789/0 conn(0x7f80b815ae20 legacy=0x7f80b80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 -- 172.21.15.114:0/3954706944 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80b0280 con 0x7f80b815b200 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b5d9b700 1 --2- 172.21.15.114:0/3954706944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098f0 tx=0x7f80a8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe40e6fc34f9d16d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b559a700 1 -- 172.21.15.114:0/3954706944 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8011830 con 0x7f80b815b200 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b559a700 1 -- 172.21.15.114:0/3954706944 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8011990 con 0x7f80b815b200 2023-11-24T06:27:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.984+0000 7f80b559a700 1 -- 172.21.15.114:0/3954706944 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8011d00 con 0x7f80b815b200 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3954706944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 msgr2=0x7f80b80b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3954706944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098f0 tx=0x7f80a8009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3954706944 shutdown_connections 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3954706944 >> v2:172.21.15.172:3300/0 conn(0x7f80b80b1a70 0x7f80b8164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3954706944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80b29a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3954706944 >> 172.21.15.114:0/3954706944 conn(0x7f80b80b57b0 msgr2=0x7f80b80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3954706944 shutdown_connections 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.985+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3954706944 wait complete. 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.986+0000 7f80bcdc9700 1 Processor -- start 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 -- start start 2023-11-24T06:27:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80b815ae20 0x7f80b81503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b814cdd0 con 0x7f80b80b1a70 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b815c290 con 0x7f80b815ae20 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80bcdc9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b814d820 con 0x7f80b815b200 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80b659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80b659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42244/0 (socket says 172.21.15.114:42244) 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.987+0000 7f80b659c700 1 -- 172.21.15.114:0/3308824259 learned_addr learned my addr 172.21.15.114:0/3308824259 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b6d9d700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b5d9b700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80b815ae20 0x7f80b81503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 -- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 msgr2=0x7f80b80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 -- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80b815ae20 msgr2=0x7f80b81503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80b815ae20 0x7f80b81503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 -- 172.21.15.114:0/3308824259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a8009550 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80b659c700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c500 tx=0x7f80ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0052d0 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac005470 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b815b680 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0038c0 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.988+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b815bc40 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.989+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b815c570 con 0x7f80b80b1a70 2023-11-24T06:27:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.989+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f80ac003a20 con 0x7f80b80b1a70 2023-11-24T06:27:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.990+0000 7f80a6ffd700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 0x7f809806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.990+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80ac088d20 con 0x7f80b80b1a70 2023-11-24T06:27:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.990+0000 7f80b5d9b700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 0x7f809806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.990+0000 7f80b5d9b700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 0x7f809806d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098f0 tx=0x7f80a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:53.995+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80ac053b80 con 0x7f80b80b1a70 2023-11-24T06:27:54.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.180+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80b80b39a0 con 0x7f809806b080 2023-11-24T06:27:54.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.187+0000 7f80a6ffd700 1 -- 172.21.15.114:0/3308824259 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289351 (secure 0 0 0) 0x7f80a8011630 con 0x7f809806b080 2023-11-24T06:27:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 msgr2=0x7f809806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 0x7f809806d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098f0 tx=0x7f80a8008030 comp rx=0 tx=0).stop 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 msgr2=0x7f80b814c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c500 tx=0x7f80ac00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 shutdown_connections 2023-11-24T06:27:54.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f809806b080 0x7f809806d540 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80b815b200 0x7f80b80d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80b80b1a70 0x7f80b814c890 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 --2- 172.21.15.114:0/3308824259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80b815ae20 0x7f80b81503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 >> 172.21.15.114:0/3308824259 conn(0x7f80b80b57b0 msgr2=0x7f80b80b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 shutdown_connections 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.192+0000 7f80bcdc9700 1 -- 172.21.15.114:0/3308824259 wait complete. 2023-11-24T06:27:54.198 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:27:54.206 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-11-24T06:27:53.754380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222852,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751868,"statfs":{"total":773094113280,"available":772865912832,"internally_reserved":0,"allocated":6823936,"data_stored":3503932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.582012"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188600+0000","last_change":"2023-11-24T06:27:45.196764+0000","last_active":"2023-11-24T06:27:46.188600+0000","last_peered":"2023-11-24T06:27:46.188600+0000","last_clean":"2023-11-24T06:27:46.188600+0000","last_became_active":"2023-11-24T06:27:45.196459+0000","last_became_peered":"2023-11-24T06:27:45.196459+0000","last_unstale":"2023-11-24T06:27:46.188600+0000","last_undegraded":"2023-11-24T06:27:46.188600+0000","last_fullsized":"2023-11-24T06:27:46.188600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:10.953646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439357+0000","last_change":"2023-11-24T06:27:45.199441+0000","last_active":"2023-11-24T06:27:46.439357+0000","last_peered":"2023-11-24T06:27:46.439357+0000","last_clean":"2023-11-24T06:27:46.439357+0000","last_became_active":"2023-11-24T06:27:45.199159+0000","last_became_peered":"2023-11-24T06:27:45.199159+0000","last_unstale":"2023-11-24T06:27:46.439357+0000","last_undegraded":"2023-11-24T06:27:46.439357+0000","last_fullsized":"2023-11-24T06:27:46.439357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:47:47.888236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192023+0000","last_change":"2023-11-24T06:27:45.185579+0000","last_active":"2023-11-24T06:27:46.192023+0000","last_peered":"2023-11-24T06:27:46.192023+0000","last_clean":"2023-11-24T06:27:46.192023+0000","last_became_active":"2023-11-24T06:27:45.185285+0000","last_became_peered":"2023-11-24T06:27:45.185285+0000","last_unstale":"2023-11-24T06:27:46.192023+0000","last_undegraded":"2023-11-24T06:27:46.192023+0000","last_fullsized":"2023-11-24T06:27:46.192023+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:18.377171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501686+0000","last_change":"2023-11-24T06:27:45.186864+0000","last_active":"2023-11-24T06:27:46.501686+0000","last_peered":"2023-11-24T06:27:46.501686+0000","last_clean":"2023-11-24T06:27:46.501686+0000","last_became_active":"2023-11-24T06:27:45.186337+0000","last_became_peered":"2023-11-24T06:27:45.186337+0000","last_unstale":"2023-11-24T06:27:46.501686+0000","last_undegraded":"2023-11-24T06:27:46.501686+0000","last_fullsized":"2023-11-24T06:27:46.501686+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:06.517029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188111+0000","last_change":"2023-11-24T06:27:45.197360+0000","last_active":"2023-11-24T06:27:46.188111+0000","last_peered":"2023-11-24T06:27:46.188111+0000","last_clean":"2023-11-24T06:27:46.188111+0000","last_became_active":"2023-11-24T06:27:45.197045+0000","last_became_peered":"2023-11-24T06:27:45.197045+0000","last_unstale":"2023-11-24T06:27:46.188111+0000","last_undegraded":"2023-11-24T06:27:46.188111+0000","last_fullsized":"2023-11-24T06:27:46.188111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:50.810177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501275+0000","last_change":"2023-11-24T06:27:45.187750+0000","last_active":"2023-11-24T06:27:46.501275+0000","last_peered":"2023-11-24T06:27:46.501275+0000","last_clean":"2023-11-24T06:27:46.501275+0000","last_became_active":"2023-11-24T06:27:45.187445+0000","last_became_peered":"2023-11-24T06:27:45.187445+0000","last_unstale":"2023-11-24T06:27:46.501275+0000","last_undegraded":"2023-11-24T06:27:46.501275+0000","last_fullsized":"2023-11-24T06:27:46.501275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:06.517029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384121+0000","last_change":"2023-11-24T06:27:45.189352+0000","last_active":"2023-11-24T06:27:46.384121+0000","last_peered":"2023-11-24T06:27:46.384121+0000","last_clean":"2023-11-24T06:27:46.384121+0000","last_became_active":"2023-11-24T06:27:45.189036+0000","last_became_peered":"2023-11-24T06:27:45.189036+0000","last_unstale":"2023-11-24T06:27:46.384121+0000","last_undegraded":"2023-11-24T06:27:46.384121+0000","last_fullsized":"2023-11-24T06:27:46.384121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:48.928133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439075+0000","last_change":"2023-11-24T06:27:45.193543+0000","last_active":"2023-11-24T06:27:46.439075+0000","last_peered":"2023-11-24T06:27:46.439075+0000","last_clean":"2023-11-24T06:27:46.439075+0000","last_became_active":"2023-11-24T06:27:45.192904+0000","last_became_peered":"2023-11-24T06:27:45.192904+0000","last_unstale":"2023-11-24T06:27:46.439075+0000","last_undegraded":"2023-11-24T06:27:46.439075+0000","last_fullsized":"2023-11-24T06:27:46.439075+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:47.980944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384458+0000","last_change":"2023-11-24T06:27:45.186031+0000","last_active":"2023-11-24T06:27:46.384458+0000","last_peered":"2023-11-24T06:27:46.384458+0000","last_clean":"2023-11-24T06:27:46.384458+0000","last_became_active":"2023-11-24T06:27:45.185737+0000","last_became_peered":"2023-11-24T06:27:45.185737+0000","last_unstale":"2023-11-24T06:27:46.384458+0000","last_undegraded":"2023-11-24T06:27:46.384458+0000","last_fullsized":"2023-11-24T06:27:46.384458+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:01.183615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501623+0000","last_change":"2023-11-24T06:27:45.195309+0000","last_active":"2023-11-24T06:27:46.501623+0000","last_peered":"2023-11-24T06:27:46.501623+0000","last_clean":"2023-11-24T06:27:46.501623+0000","last_became_active":"2023-11-24T06:27:45.194975+0000","last_became_peered":"2023-11-24T06:27:45.194975+0000","last_unstale":"2023-11-24T06:27:46.501623+0000","last_undegraded":"2023-11-24T06:27:46.501623+0000","last_fullsized":"2023-11-24T06:27:46.501623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:24.659946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501473+0000","last_change":"2023-11-24T06:27:45.195306+0000","last_active":"2023-11-24T06:27:46.501473+0000","last_peered":"2023-11-24T06:27:46.501473+0000","last_clean":"2023-11-24T06:27:46.501473+0000","last_became_active":"2023-11-24T06:27:45.194992+0000","last_became_peered":"2023-11-24T06:27:45.194992+0000","last_unstale":"2023-11-24T06:27:46.501473+0000","last_undegraded":"2023-11-24T06:27:46.501473+0000","last_fullsized":"2023-11-24T06:27:46.501473+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:16.638156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501731+0000","last_change":"2023-11-24T06:27:45.186967+0000","last_active":"2023-11-24T06:27:46.501731+0000","last_peered":"2023-11-24T06:27:46.501731+0000","last_clean":"2023-11-24T06:27:46.501731+0000","last_became_active":"2023-11-24T06:27:45.186712+0000","last_became_peered":"2023-11-24T06:27:45.186712+0000","last_unstale":"2023-11-24T06:27:46.501731+0000","last_undegraded":"2023-11-24T06:27:46.501731+0000","last_fullsized":"2023-11-24T06:27:46.501731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:28.930496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191696+0000","last_change":"2023-11-24T06:27:45.196074+0000","last_active":"2023-11-24T06:27:46.191696+0000","last_peered":"2023-11-24T06:27:46.191696+0000","last_clean":"2023-11-24T06:27:46.191696+0000","last_became_active":"2023-11-24T06:27:45.195839+0000","last_became_peered":"2023-11-24T06:27:45.195839+0000","last_unstale":"2023-11-24T06:27:46.191696+0000","last_undegraded":"2023-11-24T06:27:46.191696+0000","last_fullsized":"2023-11-24T06:27:46.191696+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:11.639139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384221+0000","last_change":"2023-11-24T06:27:45.187909+0000","last_active":"2023-11-24T06:27:46.384221+0000","last_peered":"2023-11-24T06:27:46.384221+0000","last_clean":"2023-11-24T06:27:46.384221+0000","last_became_active":"2023-11-24T06:27:45.187581+0000","last_became_peered":"2023-11-24T06:27:45.187581+0000","last_unstale":"2023-11-24T06:27:46.384221+0000","last_undegraded":"2023-11-24T06:27:46.384221+0000","last_fullsized":"2023-11-24T06:27:46.384221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:36.941024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384142+0000","last_change":"2023-11-24T06:27:45.198410+0000","last_active":"2023-11-24T06:27:46.384142+0000","last_peered":"2023-11-24T06:27:46.384142+0000","last_clean":"2023-11-24T06:27:46.384142+0000","last_became_active":"2023-11-24T06:27:45.198111+0000","last_became_peered":"2023-11-24T06:27:45.198111+0000","last_unstale":"2023-11-24T06:27:46.384142+0000","last_undegraded":"2023-11-24T06:27:46.384142+0000","last_fullsized":"2023-11-24T06:27:46.384142+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:17:47.474910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501084+0000","last_change":"2023-11-24T06:27:45.192899+0000","last_active":"2023-11-24T06:27:46.501084+0000","last_peered":"2023-11-24T06:27:46.501084+0000","last_clean":"2023-11-24T06:27:46.501084+0000","last_became_active":"2023-11-24T06:27:45.192662+0000","last_became_peered":"2023-11-24T06:27:45.192662+0000","last_unstale":"2023-11-24T06:27:46.501084+0000","last_undegraded":"2023-11-24T06:27:46.501084+0000","last_fullsized":"2023-11-24T06:27:46.501084+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:28.618271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439465+0000","last_change":"2023-11-24T06:27:45.193370+0000","last_active":"2023-11-24T06:27:46.439465+0000","last_peered":"2023-11-24T06:27:46.439465+0000","last_clean":"2023-11-24T06:27:46.439465+0000","last_became_active":"2023-11-24T06:27:45.193080+0000","last_became_peered":"2023-11-24T06:27:45.193080+0000","last_unstale":"2023-11-24T06:27:46.439465+0000","last_undegraded":"2023-11-24T06:27:46.439465+0000","last_fullsized":"2023-11-24T06:27:46.439465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:46.645009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192399+0000","last_change":"2023-11-24T06:27:45.191530+0000","last_active":"2023-11-24T06:27:46.192399+0000","last_peered":"2023-11-24T06:27:46.192399+0000","last_clean":"2023-11-24T06:27:46.192399+0000","last_became_active":"2023-11-24T06:27:45.191233+0000","last_became_peered":"2023-11-24T06:27:45.191233+0000","last_unstale":"2023-11-24T06:27:46.192399+0000","last_undegraded":"2023-11-24T06:27:46.192399+0000","last_fullsized":"2023-11-24T06:27:46.192399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:23.809610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188370+0000","last_change":"2023-11-24T06:27:45.196827+0000","last_active":"2023-11-24T06:27:46.188370+0000","last_peered":"2023-11-24T06:27:46.188370+0000","last_clean":"2023-11-24T06:27:46.188370+0000","last_became_active":"2023-11-24T06:27:45.196474+0000","last_became_peered":"2023-11-24T06:27:45.196474+0000","last_unstale":"2023-11-24T06:27:46.188370+0000","last_undegraded":"2023-11-24T06:27:46.188370+0000","last_fullsized":"2023-11-24T06:27:46.188370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:36.560026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439320+0000","last_change":"2023-11-24T06:27:45.199695+0000","last_active":"2023-11-24T06:27:46.439320+0000","last_peered":"2023-11-24T06:27:46.439320+0000","last_clean":"2023-11-24T06:27:46.439320+0000","last_became_active":"2023-11-24T06:27:45.199046+0000","last_became_peered":"2023-11-24T06:27:45.199046+0000","last_unstale":"2023-11-24T06:27:46.439320+0000","last_undegraded":"2023-11-24T06:27:46.439320+0000","last_fullsized":"2023-11-24T06:27:46.439320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:02.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191275+0000","last_change":"2023-11-24T06:27:45.191474+0000","last_active":"2023-11-24T06:27:46.191275+0000","last_peered":"2023-11-24T06:27:46.191275+0000","last_clean":"2023-11-24T06:27:46.191275+0000","last_became_active":"2023-11-24T06:27:45.191171+0000","last_became_peered":"2023-11-24T06:27:45.191171+0000","last_unstale":"2023-11-24T06:27:46.191275+0000","last_undegraded":"2023-11-24T06:27:46.191275+0000","last_fullsized":"2023-11-24T06:27:46.191275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:28.930496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439113+0000","last_change":"2023-11-24T06:27:45.196449+0000","last_active":"2023-11-24T06:27:46.439113+0000","last_peered":"2023-11-24T06:27:46.439113+0000","last_clean":"2023-11-24T06:27:46.439113+0000","last_became_active":"2023-11-24T06:27:45.196184+0000","last_became_peered":"2023-11-24T06:27:45.196184+0000","last_unstale":"2023-11-24T06:27:46.439113+0000","last_undegraded":"2023-11-24T06:27:46.439113+0000","last_fullsized":"2023-11-24T06:27:46.439113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:14.864903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191525+0000","last_change":"2023-11-24T06:27:45.199577+0000","last_active":"2023-11-24T06:27:46.191525+0000","last_peered":"2023-11-24T06:27:46.191525+0000","last_clean":"2023-11-24T06:27:46.191525+0000","last_became_active":"2023-11-24T06:27:45.199274+0000","last_became_peered":"2023-11-24T06:27:45.199274+0000","last_unstale":"2023-11-24T06:27:46.191525+0000","last_undegraded":"2023-11-24T06:27:46.191525+0000","last_fullsized":"2023-11-24T06:27:46.191525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:06.517029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191311+0000","last_change":"2023-11-24T06:27:45.196259+0000","last_active":"2023-11-24T06:27:46.191311+0000","last_peered":"2023-11-24T06:27:46.191311+0000","last_clean":"2023-11-24T06:27:46.191311+0000","last_became_active":"2023-11-24T06:27:45.196013+0000","last_became_peered":"2023-11-24T06:27:45.196013+0000","last_unstale":"2023-11-24T06:27:46.191311+0000","last_undegraded":"2023-11-24T06:27:46.191311+0000","last_fullsized":"2023-11-24T06:27:46.191311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:45:22.793262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188504+0000","last_change":"2023-11-24T06:27:45.199317+0000","last_active":"2023-11-24T06:27:46.188504+0000","last_peered":"2023-11-24T06:27:46.188504+0000","last_clean":"2023-11-24T06:27:46.188504+0000","last_became_active":"2023-11-24T06:27:45.198954+0000","last_became_peered":"2023-11-24T06:27:45.198954+0000","last_unstale":"2023-11-24T06:27:46.188504+0000","last_undegraded":"2023-11-24T06:27:46.188504+0000","last_fullsized":"2023-11-24T06:27:46.188504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:52.546554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501813+0000","last_change":"2023-11-24T06:27:45.185558+0000","last_active":"2023-11-24T06:27:46.501813+0000","last_peered":"2023-11-24T06:27:46.501813+0000","last_clean":"2023-11-24T06:27:46.501813+0000","last_became_active":"2023-11-24T06:27:45.185276+0000","last_became_peered":"2023-11-24T06:27:45.185276+0000","last_unstale":"2023-11-24T06:27:46.501813+0000","last_undegraded":"2023-11-24T06:27:46.501813+0000","last_fullsized":"2023-11-24T06:27:46.501813+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:10.419872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501543+0000","last_change":"2023-11-24T06:27:45.191845+0000","last_active":"2023-11-24T06:27:46.501543+0000","last_peered":"2023-11-24T06:27:46.501543+0000","last_clean":"2023-11-24T06:27:46.501543+0000","last_became_active":"2023-11-24T06:27:45.191565+0000","last_became_peered":"2023-11-24T06:27:45.191565+0000","last_unstale":"2023-11-24T06:27:46.501543+0000","last_undegraded":"2023-11-24T06:27:46.501543+0000","last_fullsized":"2023-11-24T06:27:46.501543+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:10.419872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.188231+0000","last_change":"2023-11-24T06:27:45.191652+0000","last_active":"2023-11-24T06:27:46.188231+0000","last_peered":"2023-11-24T06:27:46.188231+0000","last_clean":"2023-11-24T06:27:46.188231+0000","last_became_active":"2023-11-24T06:27:45.191360+0000","last_became_peered":"2023-11-24T06:27:45.191360+0000","last_unstale":"2023-11-24T06:27:46.188231+0000","last_undegraded":"2023-11-24T06:27:46.188231+0000","last_fullsized":"2023-11-24T06:27:46.188231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:17.745755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.191429+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.191159+0000","last_became_peered":"2023-11-24T06:27:45.191159+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:14:19.196091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191493+0000","last_change":"2023-11-24T06:27:45.185398+0000","last_active":"2023-11-24T06:27:46.191493+0000","last_peered":"2023-11-24T06:27:46.191493+0000","last_clean":"2023-11-24T06:27:46.191493+0000","last_became_active":"2023-11-24T06:27:45.185106+0000","last_became_peered":"2023-11-24T06:27:45.185106+0000","last_unstale":"2023-11-24T06:27:46.191493+0000","last_undegraded":"2023-11-24T06:27:46.191493+0000","last_fullsized":"2023-11-24T06:27:46.191493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.755035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.187889+0000","last_change":"2023-11-24T06:27:45.199361+0000","last_active":"2023-11-24T06:27:46.187889+0000","last_peered":"2023-11-24T06:27:46.187889+0000","last_clean":"2023-11-24T06:27:46.187889+0000","last_became_active":"2023-11-24T06:27:45.199093+0000","last_became_peered":"2023-11-24T06:27:45.199093+0000","last_unstale":"2023-11-24T06:27:46.187889+0000","last_undegraded":"2023-11-24T06:27:46.187889+0000","last_fullsized":"2023-11-24T06:27:46.187889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:20.032382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501160+0000","last_change":"2023-11-24T06:27:45.194242+0000","last_active":"2023-11-24T06:27:46.501160+0000","last_peered":"2023-11-24T06:27:46.501160+0000","last_clean":"2023-11-24T06:27:46.501160+0000","last_became_active":"2023-11-24T06:27:45.193957+0000","last_became_peered":"2023-11-24T06:27:45.193957+0000","last_unstale":"2023-11-24T06:27:46.501160+0000","last_undegraded":"2023-11-24T06:27:46.501160+0000","last_fullsized":"2023-11-24T06:27:46.501160+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.755035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501812+0000","last_change":"2023-11-24T06:27:45.187022+0000","last_active":"2023-11-24T06:27:46.501812+0000","last_peered":"2023-11-24T06:27:46.501812+0000","last_clean":"2023-11-24T06:27:46.501812+0000","last_became_active":"2023-11-24T06:27:45.186746+0000","last_became_peered":"2023-11-24T06:27:45.186746+0000","last_unstale":"2023-11-24T06:27:46.501812+0000","last_undegraded":"2023-11-24T06:27:46.501812+0000","last_fullsized":"2023-11-24T06:27:46.501812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:07.653354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212303+0000","last_change":"2023-11-24T06:27:43.887983+0000","last_active":"2023-11-24T06:27:45.212303+0000","last_peered":"2023-11-24T06:27:45.212303+0000","last_clean":"2023-11-24T06:27:45.212303+0000","last_became_active":"2023-11-24T06:27:43.887687+0000","last_became_peered":"2023-11-24T06:27:43.887687+0000","last_unstale":"2023-11-24T06:27:45.212303+0000","last_undegraded":"2023-11-24T06:27:45.212303+0000","last_fullsized":"2023-11-24T06:27:45.212303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:47.926537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501675+0000","last_change":"2023-11-24T06:27:45.189618+0000","last_active":"2023-11-24T06:27:46.501675+0000","last_peered":"2023-11-24T06:27:46.501675+0000","last_clean":"2023-11-24T06:27:46.501675+0000","last_became_active":"2023-11-24T06:27:45.189109+0000","last_became_peered":"2023-11-24T06:27:45.189109+0000","last_unstale":"2023-11-24T06:27:46.501675+0000","last_undegraded":"2023-11-24T06:27:46.501675+0000","last_fullsized":"2023-11-24T06:27:46.501675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:39.053102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207909+0000","last_change":"2023-11-24T06:27:43.887478+0000","last_active":"2023-11-24T06:27:45.207909+0000","last_peered":"2023-11-24T06:27:45.207909+0000","last_clean":"2023-11-24T06:27:45.207909+0000","last_became_active":"2023-11-24T06:27:43.886904+0000","last_became_peered":"2023-11-24T06:27:43.886904+0000","last_unstale":"2023-11-24T06:27:45.207909+0000","last_undegraded":"2023-11-24T06:27:45.207909+0000","last_fullsized":"2023-11-24T06:27:45.207909+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:57.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501874+0000","last_change":"2023-11-24T06:27:45.184656+0000","last_active":"2023-11-24T06:27:46.501874+0000","last_peered":"2023-11-24T06:27:46.501874+0000","last_clean":"2023-11-24T06:27:46.501874+0000","last_became_active":"2023-11-24T06:27:45.184372+0000","last_became_peered":"2023-11-24T06:27:45.184372+0000","last_unstale":"2023-11-24T06:27:46.501874+0000","last_undegraded":"2023-11-24T06:27:46.501874+0000","last_fullsized":"2023-11-24T06:27:46.501874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:43.478543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209895+0000","last_change":"2023-11-24T06:27:43.893948+0000","last_active":"2023-11-24T06:27:45.209895+0000","last_peered":"2023-11-24T06:27:45.209895+0000","last_clean":"2023-11-24T06:27:45.209895+0000","last_became_active":"2023-11-24T06:27:43.893661+0000","last_became_peered":"2023-11-24T06:27:43.893661+0000","last_unstale":"2023-11-24T06:27:45.209895+0000","last_undegraded":"2023-11-24T06:27:45.209895+0000","last_fullsized":"2023-11-24T06:27:45.209895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:38:51.432906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501588+0000","last_change":"2023-11-24T06:27:45.189667+0000","last_active":"2023-11-24T06:27:46.501588+0000","last_peered":"2023-11-24T06:27:46.501588+0000","last_clean":"2023-11-24T06:27:46.501588+0000","last_became_active":"2023-11-24T06:27:45.189394+0000","last_became_peered":"2023-11-24T06:27:45.189394+0000","last_unstale":"2023-11-24T06:27:46.501588+0000","last_undegraded":"2023-11-24T06:27:46.501588+0000","last_fullsized":"2023-11-24T06:27:46.501588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:18.377171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207708+0000","last_change":"2023-11-24T06:27:43.884164+0000","last_active":"2023-11-24T06:27:45.207708+0000","last_peered":"2023-11-24T06:27:45.207708+0000","last_clean":"2023-11-24T06:27:45.207708+0000","last_became_active":"2023-11-24T06:27:43.883886+0000","last_became_peered":"2023-11-24T06:27:43.883886+0000","last_unstale":"2023-11-24T06:27:45.207708+0000","last_undegraded":"2023-11-24T06:27:45.207708+0000","last_fullsized":"2023-11-24T06:27:45.207708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:37:32.615431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439144+0000","last_change":"2023-11-24T06:27:45.196404+0000","last_active":"2023-11-24T06:27:46.439144+0000","last_peered":"2023-11-24T06:27:46.439144+0000","last_clean":"2023-11-24T06:27:46.439144+0000","last_became_active":"2023-11-24T06:27:45.196180+0000","last_became_peered":"2023-11-24T06:27:45.196180+0000","last_unstale":"2023-11-24T06:27:46.439144+0000","last_undegraded":"2023-11-24T06:27:46.439144+0000","last_fullsized":"2023-11-24T06:27:46.439144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:01.939068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207774+0000","last_change":"2023-11-24T06:27:43.887860+0000","last_active":"2023-11-24T06:27:45.207774+0000","last_peered":"2023-11-24T06:27:45.207774+0000","last_clean":"2023-11-24T06:27:45.207774+0000","last_became_active":"2023-11-24T06:27:43.887522+0000","last_became_peered":"2023-11-24T06:27:43.887522+0000","last_unstale":"2023-11-24T06:27:45.207774+0000","last_undegraded":"2023-11-24T06:27:45.207774+0000","last_fullsized":"2023-11-24T06:27:45.207774+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:23.658351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439483+0000","last_change":"2023-11-24T06:27:45.190694+0000","last_active":"2023-11-24T06:27:46.439483+0000","last_peered":"2023-11-24T06:27:46.439483+0000","last_clean":"2023-11-24T06:27:46.439483+0000","last_became_active":"2023-11-24T06:27:45.190410+0000","last_became_peered":"2023-11-24T06:27:45.190410+0000","last_unstale":"2023-11-24T06:27:46.439483+0000","last_undegraded":"2023-11-24T06:27:46.439483+0000","last_fullsized":"2023-11-24T06:27:46.439483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:55.059669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208021+0000","last_change":"2023-11-24T06:27:43.888671+0000","last_active":"2023-11-24T06:27:45.208021+0000","last_peered":"2023-11-24T06:27:45.208021+0000","last_clean":"2023-11-24T06:27:45.208021+0000","last_became_active":"2023-11-24T06:27:43.888398+0000","last_became_peered":"2023-11-24T06:27:43.888398+0000","last_unstale":"2023-11-24T06:27:45.208021+0000","last_undegraded":"2023-11-24T06:27:45.208021+0000","last_fullsized":"2023-11-24T06:27:45.208021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:04.828359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191470+0000","last_change":"2023-11-24T06:27:45.198015+0000","last_active":"2023-11-24T06:27:46.191470+0000","last_peered":"2023-11-24T06:27:46.191470+0000","last_clean":"2023-11-24T06:27:46.191470+0000","last_became_active":"2023-11-24T06:27:45.197735+0000","last_became_peered":"2023-11-24T06:27:45.197735+0000","last_unstale":"2023-11-24T06:27:46.191470+0000","last_undegraded":"2023-11-24T06:27:46.191470+0000","last_fullsized":"2023-11-24T06:27:46.191470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:02:57.950709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179413+0000","last_change":"2023-11-24T06:27:43.888597+0000","last_active":"2023-11-24T06:27:45.179413+0000","last_peered":"2023-11-24T06:27:45.179413+0000","last_clean":"2023-11-24T06:27:45.179413+0000","last_became_active":"2023-11-24T06:27:43.888055+0000","last_became_peered":"2023-11-24T06:27:43.888055+0000","last_unstale":"2023-11-24T06:27:45.179413+0000","last_undegraded":"2023-11-24T06:27:45.179413+0000","last_fullsized":"2023-11-24T06:27:45.179413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:52.194056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439166+0000","last_change":"2023-11-24T06:27:45.194260+0000","last_active":"2023-11-24T06:27:46.439166+0000","last_peered":"2023-11-24T06:27:46.439166+0000","last_clean":"2023-11-24T06:27:46.439166+0000","last_became_active":"2023-11-24T06:27:45.193685+0000","last_became_peered":"2023-11-24T06:27:45.193685+0000","last_unstale":"2023-11-24T06:27:46.439166+0000","last_undegraded":"2023-11-24T06:27:46.439166+0000","last_fullsized":"2023-11-24T06:27:46.439166+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:56.547263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179464+0000","last_change":"2023-11-24T06:27:43.890543+0000","last_active":"2023-11-24T06:27:45.179464+0000","last_peered":"2023-11-24T06:27:45.179464+0000","last_clean":"2023-11-24T06:27:45.179464+0000","last_became_active":"2023-11-24T06:27:43.890213+0000","last_became_peered":"2023-11-24T06:27:43.890213+0000","last_unstale":"2023-11-24T06:27:45.179464+0000","last_undegraded":"2023-11-24T06:27:45.179464+0000","last_fullsized":"2023-11-24T06:27:45.179464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:07.967959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384560+0000","last_change":"2023-11-24T06:27:45.198996+0000","last_active":"2023-11-24T06:27:46.384560+0000","last_peered":"2023-11-24T06:27:46.384560+0000","last_clean":"2023-11-24T06:27:46.384560+0000","last_became_active":"2023-11-24T06:27:45.198728+0000","last_became_peered":"2023-11-24T06:27:45.198728+0000","last_unstale":"2023-11-24T06:27:46.384560+0000","last_undegraded":"2023-11-24T06:27:46.384560+0000","last_fullsized":"2023-11-24T06:27:46.384560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:02.415473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.181326+0000","last_change":"2023-11-24T06:27:43.888862+0000","last_active":"2023-11-24T06:27:45.181326+0000","last_peered":"2023-11-24T06:27:45.181326+0000","last_clean":"2023-11-24T06:27:45.181326+0000","last_became_active":"2023-11-24T06:27:43.888560+0000","last_became_peered":"2023-11-24T06:27:43.888560+0000","last_unstale":"2023-11-24T06:27:45.181326+0000","last_undegraded":"2023-11-24T06:27:45.181326+0000","last_fullsized":"2023-11-24T06:27:45.181326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:51:52.194056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192239+0000","last_change":"2023-11-24T06:27:45.198313+0000","last_active":"2023-11-24T06:27:46.192239+0000","last_peered":"2023-11-24T06:27:46.192239+0000","last_clean":"2023-11-24T06:27:46.192239+0000","last_became_active":"2023-11-24T06:27:45.197887+0000","last_became_peered":"2023-11-24T06:27:45.197887+0000","last_unstale":"2023-11-24T06:27:46.192239+0000","last_undegraded":"2023-11-24T06:27:46.192239+0000","last_fullsized":"2023-11-24T06:27:46.192239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:43.478543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.187762+0000","last_change":"2023-11-24T06:27:43.886888+0000","last_active":"2023-11-24T06:27:45.187762+0000","last_peered":"2023-11-24T06:27:45.187762+0000","last_clean":"2023-11-24T06:27:45.187762+0000","last_became_active":"2023-11-24T06:27:43.886584+0000","last_became_peered":"2023-11-24T06:27:43.886584+0000","last_unstale":"2023-11-24T06:27:45.187762+0000","last_undegraded":"2023-11-24T06:27:45.187762+0000","last_fullsized":"2023-11-24T06:27:45.187762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:17:04.535818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501263+0000","last_change":"2023-11-24T06:27:45.187309+0000","last_active":"2023-11-24T06:27:46.501263+0000","last_peered":"2023-11-24T06:27:46.501263+0000","last_clean":"2023-11-24T06:27:46.501263+0000","last_became_active":"2023-11-24T06:27:45.187073+0000","last_became_peered":"2023-11-24T06:27:45.187073+0000","last_unstale":"2023-11-24T06:27:46.501263+0000","last_undegraded":"2023-11-24T06:27:46.501263+0000","last_fullsized":"2023-11-24T06:27:46.501263+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:41.101663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209853+0000","last_change":"2023-11-24T06:27:43.887129+0000","last_active":"2023-11-24T06:27:45.209853+0000","last_peered":"2023-11-24T06:27:45.209853+0000","last_clean":"2023-11-24T06:27:45.209853+0000","last_became_active":"2023-11-24T06:27:43.886873+0000","last_became_peered":"2023-11-24T06:27:43.886873+0000","last_unstale":"2023-11-24T06:27:45.209853+0000","last_undegraded":"2023-11-24T06:27:45.209853+0000","last_fullsized":"2023-11-24T06:27:45.209853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.841385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192117+0000","last_change":"2023-11-24T06:27:45.195391+0000","last_active":"2023-11-24T06:27:46.192117+0000","last_peered":"2023-11-24T06:27:46.192117+0000","last_clean":"2023-11-24T06:27:46.192117+0000","last_became_active":"2023-11-24T06:27:45.195147+0000","last_became_peered":"2023-11-24T06:27:45.195147+0000","last_unstale":"2023-11-24T06:27:46.192117+0000","last_undegraded":"2023-11-24T06:27:46.192117+0000","last_fullsized":"2023-11-24T06:27:46.192117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:51.039635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.213669+0000","last_change":"2023-11-24T06:27:43.891037+0000","last_active":"2023-11-24T06:27:45.213669+0000","last_peered":"2023-11-24T06:27:45.213669+0000","last_clean":"2023-11-24T06:27:45.213669+0000","last_became_active":"2023-11-24T06:27:43.890756+0000","last_became_peered":"2023-11-24T06:27:43.890756+0000","last_unstale":"2023-11-24T06:27:45.213669+0000","last_undegraded":"2023-11-24T06:27:45.213669+0000","last_fullsized":"2023-11-24T06:27:45.213669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:38.051507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501491+0000","last_change":"2023-11-24T06:27:45.199525+0000","last_active":"2023-11-24T06:27:46.501491+0000","last_peered":"2023-11-24T06:27:46.501491+0000","last_clean":"2023-11-24T06:27:46.501491+0000","last_became_active":"2023-11-24T06:27:45.199300+0000","last_became_peered":"2023-11-24T06:27:45.199300+0000","last_unstale":"2023-11-24T06:27:46.501491+0000","last_undegraded":"2023-11-24T06:27:46.501491+0000","last_fullsized":"2023-11-24T06:27:46.501491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:44.755035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209609+0000","last_change":"2023-11-24T06:27:43.887228+0000","last_active":"2023-11-24T06:27:45.209609+0000","last_peered":"2023-11-24T06:27:45.209609+0000","last_clean":"2023-11-24T06:27:45.209609+0000","last_became_active":"2023-11-24T06:27:43.886820+0000","last_became_peered":"2023-11-24T06:27:43.886820+0000","last_unstale":"2023-11-24T06:27:45.209609+0000","last_undegraded":"2023-11-24T06:27:45.209609+0000","last_fullsized":"2023-11-24T06:27:45.209609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:47.443503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191576+0000","last_change":"2023-11-24T06:27:45.194557+0000","last_active":"2023-11-24T06:27:46.191576+0000","last_peered":"2023-11-24T06:27:46.191576+0000","last_clean":"2023-11-24T06:27:46.191576+0000","last_became_active":"2023-11-24T06:27:45.192759+0000","last_became_peered":"2023-11-24T06:27:45.192759+0000","last_unstale":"2023-11-24T06:27:46.191576+0000","last_undegraded":"2023-11-24T06:27:46.191576+0000","last_fullsized":"2023-11-24T06:27:46.191576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:56.389229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.185787+0000","last_change":"2023-11-24T06:27:43.880061+0000","last_active":"2023-11-24T06:27:45.185787+0000","last_peered":"2023-11-24T06:27:45.185787+0000","last_clean":"2023-11-24T06:27:45.185787+0000","last_became_active":"2023-11-24T06:27:43.879790+0000","last_became_peered":"2023-11-24T06:27:43.879790+0000","last_unstale":"2023-11-24T06:27:45.185787+0000","last_undegraded":"2023-11-24T06:27:45.185787+0000","last_fullsized":"2023-11-24T06:27:45.185787+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:09.952050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.384056+0000","last_change":"2023-11-24T06:27:45.187007+0000","last_active":"2023-11-24T06:27:46.384056+0000","last_peered":"2023-11-24T06:27:46.384056+0000","last_clean":"2023-11-24T06:27:46.384056+0000","last_became_active":"2023-11-24T06:27:45.186745+0000","last_became_peered":"2023-11-24T06:27:45.186745+0000","last_unstale":"2023-11-24T06:27:46.384056+0000","last_undegraded":"2023-11-24T06:27:46.384056+0000","last_fullsized":"2023-11-24T06:27:46.384056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:26.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179158+0000","last_change":"2023-11-24T06:27:43.881814+0000","last_active":"2023-11-24T06:27:45.179158+0000","last_peered":"2023-11-24T06:27:45.179158+0000","last_clean":"2023-11-24T06:27:45.179158+0000","last_became_active":"2023-11-24T06:27:43.881477+0000","last_became_peered":"2023-11-24T06:27:43.881477+0000","last_unstale":"2023-11-24T06:27:45.179158+0000","last_undegraded":"2023-11-24T06:27:45.179158+0000","last_fullsized":"2023-11-24T06:27:45.179158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:02.376935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191280+0000","last_change":"2023-11-24T06:27:45.199368+0000","last_active":"2023-11-24T06:27:46.191280+0000","last_peered":"2023-11-24T06:27:46.191280+0000","last_clean":"2023-11-24T06:27:46.191280+0000","last_became_active":"2023-11-24T06:27:45.199103+0000","last_became_peered":"2023-11-24T06:27:45.199103+0000","last_unstale":"2023-11-24T06:27:46.191280+0000","last_undegraded":"2023-11-24T06:27:46.191280+0000","last_fullsized":"2023-11-24T06:27:46.191280+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:28.618271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212369+0000","last_change":"2023-11-24T06:27:43.885895+0000","last_active":"2023-11-24T06:27:45.212369+0000","last_peered":"2023-11-24T06:27:45.212369+0000","last_clean":"2023-11-24T06:27:45.212369+0000","last_became_active":"2023-11-24T06:27:43.885623+0000","last_became_peered":"2023-11-24T06:27:43.885623+0000","last_unstale":"2023-11-24T06:27:45.212369+0000","last_undegraded":"2023-11-24T06:27:45.212369+0000","last_fullsized":"2023-11-24T06:27:45.212369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:35.939429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192409+0000","last_change":"2023-11-24T06:27:45.191080+0000","last_active":"2023-11-24T06:27:46.192409+0000","last_peered":"2023-11-24T06:27:46.192409+0000","last_clean":"2023-11-24T06:27:46.192409+0000","last_became_active":"2023-11-24T06:27:45.190816+0000","last_became_peered":"2023-11-24T06:27:45.190816+0000","last_unstale":"2023-11-24T06:27:46.192409+0000","last_undegraded":"2023-11-24T06:27:46.192409+0000","last_fullsized":"2023-11-24T06:27:46.192409+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:48.445098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.209513+0000","last_change":"2023-11-24T06:27:43.887376+0000","last_active":"2023-11-24T06:27:45.209513+0000","last_peered":"2023-11-24T06:27:45.209513+0000","last_clean":"2023-11-24T06:27:45.209513+0000","last_became_active":"2023-11-24T06:27:43.887125+0000","last_became_peered":"2023-11-24T06:27:43.887125+0000","last_unstale":"2023-11-24T06:27:45.209513+0000","last_undegraded":"2023-11-24T06:27:45.209513+0000","last_fullsized":"2023-11-24T06:27:45.209513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:06.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501916+0000","last_change":"2023-11-24T06:27:45.198812+0000","last_active":"2023-11-24T06:27:46.501916+0000","last_peered":"2023-11-24T06:27:46.501916+0000","last_clean":"2023-11-24T06:27:46.501916+0000","last_became_active":"2023-11-24T06:27:45.198547+0000","last_became_peered":"2023-11-24T06:27:45.198547+0000","last_unstale":"2023-11-24T06:27:46.501916+0000","last_undegraded":"2023-11-24T06:27:46.501916+0000","last_fullsized":"2023-11-24T06:27:46.501916+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:51.039635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208265+0000","last_change":"2023-11-24T06:27:43.887382+0000","last_active":"2023-11-24T06:27:45.208265+0000","last_peered":"2023-11-24T06:27:45.208265+0000","last_clean":"2023-11-24T06:27:45.208265+0000","last_became_active":"2023-11-24T06:27:43.887150+0000","last_became_peered":"2023-11-24T06:27:43.887150+0000","last_unstale":"2023-11-24T06:27:45.208265+0000","last_undegraded":"2023-11-24T06:27:45.208265+0000","last_fullsized":"2023-11-24T06:27:45.208265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.841385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439335+0000","last_change":"2023-11-24T06:27:45.197917+0000","last_active":"2023-11-24T06:27:46.439335+0000","last_peered":"2023-11-24T06:27:46.439335+0000","last_clean":"2023-11-24T06:27:46.439335+0000","last_became_active":"2023-11-24T06:27:45.197646+0000","last_became_peered":"2023-11-24T06:27:45.197646+0000","last_unstale":"2023-11-24T06:27:46.439335+0000","last_undegraded":"2023-11-24T06:27:46.439335+0000","last_fullsized":"2023-11-24T06:27:46.439335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:12.136040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.207706+0000","last_change":"2023-11-24T06:27:43.884248+0000","last_active":"2023-11-24T06:27:45.207706+0000","last_peered":"2023-11-24T06:27:45.207706+0000","last_clean":"2023-11-24T06:27:45.207706+0000","last_became_active":"2023-11-24T06:27:43.883575+0000","last_became_peered":"2023-11-24T06:27:43.883575+0000","last_unstale":"2023-11-24T06:27:45.207706+0000","last_undegraded":"2023-11-24T06:27:45.207706+0000","last_fullsized":"2023-11-24T06:27:45.207706+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:27.616676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.192202+0000","last_change":"2023-11-24T06:27:45.198775+0000","last_active":"2023-11-24T06:27:46.192202+0000","last_peered":"2023-11-24T06:27:46.192202+0000","last_clean":"2023-11-24T06:27:46.192202+0000","last_became_active":"2023-11-24T06:27:45.198554+0000","last_became_peered":"2023-11-24T06:27:45.198554+0000","last_unstale":"2023-11-24T06:27:46.192202+0000","last_undegraded":"2023-11-24T06:27:46.192202+0000","last_fullsized":"2023-11-24T06:27:46.192202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:07.653354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179085+0000","last_change":"2023-11-24T06:27:43.890800+0000","last_active":"2023-11-24T06:27:45.179085+0000","last_peered":"2023-11-24T06:27:45.179085+0000","last_clean":"2023-11-24T06:27:45.179085+0000","last_became_active":"2023-11-24T06:27:43.890508+0000","last_became_peered":"2023-11-24T06:27:43.890508+0000","last_unstale":"2023-11-24T06:27:45.179085+0000","last_undegraded":"2023-11-24T06:27:45.179085+0000","last_fullsized":"2023-11-24T06:27:45.179085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:04.795474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501710+0000","last_change":"2023-11-24T06:27:45.198840+0000","last_active":"2023-11-24T06:27:46.501710+0000","last_peered":"2023-11-24T06:27:46.501710+0000","last_clean":"2023-11-24T06:27:46.501710+0000","last_became_active":"2023-11-24T06:27:45.198523+0000","last_became_peered":"2023-11-24T06:27:45.198523+0000","last_unstale":"2023-11-24T06:27:46.501710+0000","last_undegraded":"2023-11-24T06:27:46.501710+0000","last_fullsized":"2023-11-24T06:27:46.501710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:56.389229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.212410+0000","last_change":"2023-11-24T06:27:43.888453+0000","last_active":"2023-11-24T06:27:45.212410+0000","last_peered":"2023-11-24T06:27:45.212410+0000","last_clean":"2023-11-24T06:27:45.212410+0000","last_became_active":"2023-11-24T06:27:43.888169+0000","last_became_peered":"2023-11-24T06:27:43.888169+0000","last_unstale":"2023-11-24T06:27:45.212410+0000","last_undegraded":"2023-11-24T06:27:45.212410+0000","last_fullsized":"2023-11-24T06:27:45.212410+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:37.661869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501526+0000","last_change":"2023-11-24T06:27:45.192208+0000","last_active":"2023-11-24T06:27:46.501526+0000","last_peered":"2023-11-24T06:27:46.501526+0000","last_clean":"2023-11-24T06:27:46.501526+0000","last_became_active":"2023-11-24T06:27:45.191906+0000","last_became_peered":"2023-11-24T06:27:45.191906+0000","last_unstale":"2023-11-24T06:27:46.501526+0000","last_undegraded":"2023-11-24T06:27:46.501526+0000","last_fullsized":"2023-11-24T06:27:46.501526+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:28.930496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.174123+0000","last_change":"2023-11-24T06:27:43.886183+0000","last_active":"2023-11-24T06:27:45.174123+0000","last_peered":"2023-11-24T06:27:45.174123+0000","last_clean":"2023-11-24T06:27:45.174123+0000","last_became_active":"2023-11-24T06:27:43.885892+0000","last_became_peered":"2023-11-24T06:27:43.885892+0000","last_unstale":"2023-11-24T06:27:45.174123+0000","last_undegraded":"2023-11-24T06:27:45.174123+0000","last_fullsized":"2023-11-24T06:27:45.174123+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:39.853561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501311+0000","last_change":"2023-11-24T06:27:45.187149+0000","last_active":"2023-11-24T06:27:46.501311+0000","last_peered":"2023-11-24T06:27:46.501311+0000","last_clean":"2023-11-24T06:27:46.501311+0000","last_became_active":"2023-11-24T06:27:45.186822+0000","last_became_peered":"2023-11-24T06:27:45.186822+0000","last_unstale":"2023-11-24T06:27:46.501311+0000","last_undegraded":"2023-11-24T06:27:46.501311+0000","last_fullsized":"2023-11-24T06:27:46.501311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:20:56.389229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179260+0000","last_change":"2023-11-24T06:27:43.890788+0000","last_active":"2023-11-24T06:27:45.179260+0000","last_peered":"2023-11-24T06:27:45.179260+0000","last_clean":"2023-11-24T06:27:45.179260+0000","last_became_active":"2023-11-24T06:27:43.890453+0000","last_became_peered":"2023-11-24T06:27:43.890453+0000","last_unstale":"2023-11-24T06:27:45.179260+0000","last_undegraded":"2023-11-24T06:27:45.179260+0000","last_fullsized":"2023-11-24T06:27:45.179260+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:07.841385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501843+0000","last_change":"2023-11-24T06:27:45.186658+0000","last_active":"2023-11-24T06:27:46.501843+0000","last_peered":"2023-11-24T06:27:46.501843+0000","last_clean":"2023-11-24T06:27:46.501843+0000","last_became_active":"2023-11-24T06:27:45.186365+0000","last_became_peered":"2023-11-24T06:27:45.186365+0000","last_unstale":"2023-11-24T06:27:46.501843+0000","last_undegraded":"2023-11-24T06:27:46.501843+0000","last_fullsized":"2023-11-24T06:27:46.501843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:18.377171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.208599+0000","last_change":"2023-11-24T06:27:43.889734+0000","last_active":"2023-11-24T06:27:45.208599+0000","last_peered":"2023-11-24T06:27:45.208599+0000","last_clean":"2023-11-24T06:27:45.208599+0000","last_became_active":"2023-11-24T06:27:43.889470+0000","last_became_peered":"2023-11-24T06:27:43.889470+0000","last_unstale":"2023-11-24T06:27:45.208599+0000","last_undegraded":"2023-11-24T06:27:45.208599+0000","last_fullsized":"2023-11-24T06:27:45.208599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:18.289207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501944+0000","last_change":"2023-11-24T06:27:45.192417+0000","last_active":"2023-11-24T06:27:46.501944+0000","last_peered":"2023-11-24T06:27:46.501944+0000","last_clean":"2023-11-24T06:27:46.501944+0000","last_became_active":"2023-11-24T06:27:45.192167+0000","last_became_peered":"2023-11-24T06:27:45.192167+0000","last_unstale":"2023-11-24T06:27:46.501944+0000","last_undegraded":"2023-11-24T06:27:46.501944+0000","last_fullsized":"2023-11-24T06:27:46.501944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:07.653354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501897+0000","last_change":"2023-11-24T06:27:43.889511+0000","last_active":"2023-11-24T06:27:46.501897+0000","last_peered":"2023-11-24T06:27:46.501897+0000","last_clean":"2023-11-24T06:27:46.501897+0000","last_became_active":"2023-11-24T06:27:43.889234+0000","last_became_peered":"2023-11-24T06:27:43.889234+0000","last_unstale":"2023-11-24T06:27:46.501897+0000","last_undegraded":"2023-11-24T06:27:46.501897+0000","last_fullsized":"2023-11-24T06:27:46.501897+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:01.413877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501307+0000","last_change":"2023-11-24T06:27:45.187297+0000","last_active":"2023-11-24T06:27:46.501307+0000","last_peered":"2023-11-24T06:27:46.501307+0000","last_clean":"2023-11-24T06:27:46.501307+0000","last_became_active":"2023-11-24T06:27:45.186978+0000","last_became_peered":"2023-11-24T06:27:45.186978+0000","last_unstale":"2023-11-24T06:27:46.501307+0000","last_undegraded":"2023-11-24T06:27:46.501307+0000","last_fullsized":"2023-11-24T06:27:46.501307+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:08.969554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.188357+0000","last_change":"2023-11-24T06:27:43.886417+0000","last_active":"2023-11-24T06:27:45.188357+0000","last_peered":"2023-11-24T06:27:45.188357+0000","last_clean":"2023-11-24T06:27:45.188357+0000","last_became_active":"2023-11-24T06:27:43.886110+0000","last_became_peered":"2023-11-24T06:27:43.886110+0000","last_unstale":"2023-11-24T06:27:45.188357+0000","last_undegraded":"2023-11-24T06:27:45.188357+0000","last_fullsized":"2023-11-24T06:27:45.188357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:01.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501695+0000","last_change":"2023-11-24T06:27:45.184597+0000","last_active":"2023-11-24T06:27:46.501695+0000","last_peered":"2023-11-24T06:27:46.501695+0000","last_clean":"2023-11-24T06:27:46.501695+0000","last_became_active":"2023-11-24T06:27:45.184299+0000","last_became_peered":"2023-11-24T06:27:45.184299+0000","last_unstale":"2023-11-24T06:27:46.501695+0000","last_undegraded":"2023-11-24T06:27:46.501695+0000","last_fullsized":"2023-11-24T06:27:46.501695+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:51.039635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179445+0000","last_change":"2023-11-24T06:27:43.887202+0000","last_active":"2023-11-24T06:27:45.179445+0000","last_peered":"2023-11-24T06:27:45.179445+0000","last_clean":"2023-11-24T06:27:45.179445+0000","last_became_active":"2023-11-24T06:27:43.886926+0000","last_became_peered":"2023-11-24T06:27:43.886926+0000","last_unstale":"2023-11-24T06:27:45.179445+0000","last_undegraded":"2023-11-24T06:27:45.179445+0000","last_fullsized":"2023-11-24T06:27:45.179445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:04.828359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439423+0000","last_change":"2023-11-24T06:27:45.199563+0000","last_active":"2023-11-24T06:27:46.439423+0000","last_peered":"2023-11-24T06:27:46.439423+0000","last_clean":"2023-11-24T06:27:46.439423+0000","last_became_active":"2023-11-24T06:27:45.199267+0000","last_became_peered":"2023-11-24T06:27:45.199267+0000","last_unstale":"2023-11-24T06:27:46.439423+0000","last_undegraded":"2023-11-24T06:27:46.439423+0000","last_fullsized":"2023-11-24T06:27:46.439423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:45.747801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.174117+0000","last_change":"2023-11-24T06:27:43.885639+0000","last_active":"2023-11-24T06:27:45.174117+0000","last_peered":"2023-11-24T06:27:45.174117+0000","last_clean":"2023-11-24T06:27:45.174117+0000","last_became_active":"2023-11-24T06:27:43.885321+0000","last_became_peered":"2023-11-24T06:27:43.885321+0000","last_unstale":"2023-11-24T06:27:45.174117+0000","last_undegraded":"2023-11-24T06:27:45.174117+0000","last_fullsized":"2023-11-24T06:27:45.174117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:32.683727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.191748+0000","last_change":"2023-11-24T06:27:45.194755+0000","last_active":"2023-11-24T06:27:46.191748+0000","last_peered":"2023-11-24T06:27:46.191748+0000","last_clean":"2023-11-24T06:27:46.191748+0000","last_became_active":"2023-11-24T06:27:45.194281+0000","last_became_peered":"2023-11-24T06:27:45.194281+0000","last_unstale":"2023-11-24T06:27:46.191748+0000","last_undegraded":"2023-11-24T06:27:46.191748+0000","last_fullsized":"2023-11-24T06:27:46.191748+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:10.419872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.179699+0000","last_change":"2023-11-24T06:27:43.888919+0000","last_active":"2023-11-24T06:27:45.179699+0000","last_peered":"2023-11-24T06:27:45.179699+0000","last_clean":"2023-11-24T06:27:45.179699+0000","last_became_active":"2023-11-24T06:27:43.888593+0000","last_became_peered":"2023-11-24T06:27:43.888593+0000","last_unstale":"2023-11-24T06:27:45.179699+0000","last_undegraded":"2023-11-24T06:27:45.179699+0000","last_fullsized":"2023-11-24T06:27:45.179699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:15.636561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.501095+0000","last_change":"2023-11-24T06:27:45.195353+0000","last_active":"2023-11-24T06:27:46.501095+0000","last_peered":"2023-11-24T06:27:46.501095+0000","last_clean":"2023-11-24T06:27:46.501095+0000","last_became_active":"2023-11-24T06:27:45.194977+0000","last_became_peered":"2023-11-24T06:27:45.194977+0000","last_unstale":"2023-11-24T06:27:46.501095+0000","last_undegraded":"2023-11-24T06:27:46.501095+0000","last_fullsized":"2023-11-24T06:27:46.501095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:52:05.977321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.174437+0000","last_change":"2023-11-24T06:27:43.889027+0000","last_active":"2023-11-24T06:27:45.174437+0000","last_peered":"2023-11-24T06:27:45.174437+0000","last_clean":"2023-11-24T06:27:45.174437+0000","last_became_active":"2023-11-24T06:27:43.888756+0000","last_became_peered":"2023-11-24T06:27:43.888756+0000","last_unstale":"2023-11-24T06:27:45.174437+0000","last_undegraded":"2023-11-24T06:27:45.174437+0000","last_fullsized":"2023-11-24T06:27:45.174437+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:27.042047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439231+0000","last_change":"2023-11-24T06:27:45.194043+0000","last_active":"2023-11-24T06:27:46.439231+0000","last_peered":"2023-11-24T06:27:46.439231+0000","last_clean":"2023-11-24T06:27:46.439231+0000","last_became_active":"2023-11-24T06:27:45.192605+0000","last_became_peered":"2023-11-24T06:27:45.192605+0000","last_unstale":"2023-11-24T06:27:46.439231+0000","last_undegraded":"2023-11-24T06:27:46.439231+0000","last_fullsized":"2023-11-24T06:27:46.439231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:59.614993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.180294+0000","last_change":"2023-11-24T06:27:43.889915+0000","last_active":"2023-11-24T06:27:45.180294+0000","last_peered":"2023-11-24T06:27:45.180294+0000","last_clean":"2023-11-24T06:27:45.180294+0000","last_became_active":"2023-11-24T06:27:43.889576+0000","last_became_peered":"2023-11-24T06:27:43.889576+0000","last_unstale":"2023-11-24T06:27:45.180294+0000","last_undegraded":"2023-11-24T06:27:45.180294+0000","last_fullsized":"2023-11-24T06:27:45.180294+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:41:42.721820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:27:46.439124+0000","last_change":"2023-11-24T06:27:45.193323+0000","last_active":"2023-11-24T06:27:46.439124+0000","last_peered":"2023-11-24T06:27:46.439124+0000","last_clean":"2023-11-24T06:27:46.439124+0000","last_became_active":"2023-11-24T06:27:45.192548+0000","last_became_peered":"2023-11-24T06:27:45.192548+0000","last_unstale":"2023-11-24T06:27:46.439124+0000","last_undegraded":"2023-11-24T06:27:46.439124+0000","last_fullsized":"2023-11-24T06:27:46.439124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:43.868182+0000","last_clean_scrub_stamp":"2023-11-24T06:27:43.868182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:09.532953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.188162+0000","last_change":"2023-11-24T06:27:43.882518+0000","last_active":"2023-11-24T06:27:45.188162+0000","last_peered":"2023-11-24T06:27:45.188162+0000","last_clean":"2023-11-24T06:27:45.188162+0000","last_became_active":"2023-11-24T06:27:43.882224+0000","last_became_peered":"2023-11-24T06:27:43.882224+0000","last_unstale":"2023-11-24T06:27:45.188162+0000","last_undegraded":"2023-11-24T06:27:45.188162+0000","last_fullsized":"2023-11-24T06:27:45.188162+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:27:42.866586+0000","last_clean_scrub_stamp":"2023-11-24T06:27:42.866586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:39.853561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-24T06:27:45.178840+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:27:45.178840+0000","last_peered":"2023-11-24T06:27:45.178840+0000","last_clean":"2023-11-24T06:27:45.178840+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:27:45.178840+0000","last_undegraded":"2023-11-24T06:27:45.178840+0000","last_fullsized":"2023-11-24T06:27:45.178840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":880325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.77600000000000002,"15min":0.86599999999999999},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.378},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.87,"5min":1.373,"15min":1.373},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.411},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.345,"15min":1.345},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.80700000000000005,"5min":1.286,"15min":1.286},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.46800000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82599999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80800000000000005,"5min":0.874,"15min":0.874},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":1.379,"15min":1.379},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.62},"min":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.026,"15min":1.026},"last":0.52600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.433}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.37},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.27000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.307}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.57799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.315,"15min":0.313},"min":{"1min":0.20899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.254},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52100000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.29699999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51200000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58599999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":1.028,"15min":1.028},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.69599999999999995},"min":{"1min":0.45700000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91600000000000004,"5min":1.01,"15min":1.01},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96899999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.0369999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.44800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66500000000000004,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":1,"15min":1},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.27400000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.316},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.627}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":290449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.437},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.5}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:54.212 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:27:54.212 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:27:54.374 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.372+0000 7f52d089e700 1 Processor -- start 2023-11-24T06:27:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.372+0000 7f52d089e700 1 -- start start 2023-11-24T06:27:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52d089e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52cc0cc5e0 0x7f52cc0cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52d089e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52d089e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0b12f0 con 0x7f52cc15c0b0 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52d089e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0d09b0 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52d089e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0af240 con 0x7f52cc0d0500 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52cad9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52c9d9b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52cc0cc5e0 0x7f52cc0cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52cad9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:36800/0 (socket says 172.21.15.114:36800) 2023-11-24T06:27:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.373+0000 7f52cad9d700 1 -- 172.21.15.114:0/320540962 learned_addr learned my addr 172.21.15.114:0/320540962 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52cad9d700 1 -- 172.21.15.114:0/320540962 >> v2:172.21.15.172:3300/0 conn(0x7f52cc0cc5e0 msgr2=0x7f52cc0cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52cad9d700 1 --2- 172.21.15.114:0/320540962 >> v2:172.21.15.172:3300/0 conn(0x7f52cc0cc5e0 0x7f52cc0cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52cad9d700 1 -- 172.21.15.114:0/320540962 >> v1:172.21.15.172:6789/0 conn(0x7f52cc15c0b0 legacy=0x7f52cc0ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52cad9d700 1 -- 172.21.15.114:0/320540962 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0af520 con 0x7f52cc0d0500 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52cad9d700 1 --2- 172.21.15.114:0/320540962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009e50 tx=0x7f52c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6532b6bc346a006 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52c959a700 1 -- 172.21.15.114:0/320540962 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400e070 con 0x7f52cc0d0500 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52c959a700 1 -- 172.21.15.114:0/320540962 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400b910 con 0x7f52cc0d0500 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.374+0000 7f52c959a700 1 -- 172.21.15.114:0/320540962 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400bcc0 con 0x7f52cc0d0500 2023-11-24T06:27:54.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 -- 172.21.15.114:0/320540962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 msgr2=0x7f52cc0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 --2- 172.21.15.114:0/320540962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009e50 tx=0x7f52c400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 -- 172.21.15.114:0/320540962 shutdown_connections 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 --2- 172.21.15.114:0/320540962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc0d0500 0x7f52cc0b1a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 --2- 172.21.15.114:0/320540962 >> v2:172.21.15.172:3300/0 conn(0x7f52cc0cc5e0 0x7f52cc0cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 -- 172.21.15.114:0/320540962 >> 172.21.15.114:0/320540962 conn(0x7f52cc09aa00 msgr2=0x7f52cc09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 -- 172.21.15.114:0/320540962 shutdown_connections 2023-11-24T06:27:54.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.375+0000 7f52d089e700 1 -- 172.21.15.114:0/320540962 wait complete. 2023-11-24T06:27:54.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.376+0000 7f52d089e700 1 Processor -- start 2023-11-24T06:27:54.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.376+0000 7f52d089e700 1 -- start start 2023-11-24T06:27:54.381 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 0x7f52cc0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 0x7f52cc0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1568d0 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0cadb0 con 0x7f52cc0d0500 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52d089e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0ca1a0 con 0x7f52cc15c0b0 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42258/0 (socket says 172.21.15.114:42258) 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 -- 172.21.15.114:0/1300463893 learned_addr learned my addr 172.21.15.114:0/1300463893 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52c9d9b700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 0x7f52cc0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52cad9d700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 0x7f52cc0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 -- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 msgr2=0x7f52cc0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.382 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 0x7f52cc0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 -- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 msgr2=0x7f52cc0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 0x7f52cc0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52ca59c700 1 -- 172.21.15.114:0/1300463893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009b30 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52cad9d700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 0x7f52cc0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.377+0000 7f52c9d9b700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 0x7f52cc0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52ca59c700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c430 tx=0x7f52c000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00052d0 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c0005450 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc0ca480 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0006aa0 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.378+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0caa40 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.379+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc068fb0 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.379+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f52c0003680 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.380+0000 7f52baffd700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 0x7f52ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.380+0000 7f52c9d9b700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 0x7f52ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.380+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52c0088b40 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.380+0000 7f52c9d9b700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 0x7f52ac06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0c4e40 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.384+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52c0053970 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.653+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f52cc0b2600 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.655+0000 7f52baffd700 1 -- 172.21.15.114:0/1300463893 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f52c0056f90 con 0x7f52cc0cc5e0 2023-11-24T06:27:54.657 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2023-11-24T06:27:54.657 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQBVQWBljCLSOBAAaaGCKHCtrze2z1KihEG+9g== 2023-11-24T06:27:54.657 INFO:teuthology.orchestra.run.smithi114.stdout: caps mds = "allow" 2023-11-24T06:27:54.658 INFO:teuthology.orchestra.run.smithi114.stdout: caps mon = "allow r" 2023-11-24T06:27:54.658 INFO:teuthology.orchestra.run.smithi114.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:27:54.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 msgr2=0x7f52ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 0x7f52ac06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0c4e40 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 msgr2=0x7f52cc156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c430 tx=0x7f52c000c7f0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 shutdown_connections 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f52ac06b080 0x7f52ac06d540 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f52cc15c0b0 0x7f52cc0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f52cc0cc5e0 0x7f52cc156390 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 --2- 172.21.15.114:0/1300463893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52cc0d0500 0x7f52cc0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 >> 172.21.15.114:0/1300463893 conn(0x7f52cc09aa00 msgr2=0x7f52cc0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 shutdown_connections 2023-11-24T06:27:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.657+0000 7f52d089e700 1 -- 172.21.15.114:0/1300463893 wait complete. 2023-11-24T06:27:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:updated caps for client.0 2023-11-24T06:27:54.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:54.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 Processor -- start 2023-11-24T06:27:54.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 -- start start 2023-11-24T06:27:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23040cf360 0x7f23040b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 0x7f230415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230415a790 con 0x7f230415a3b0 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23040cf8b0 con 0x7f23040cf360 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.791+0000 7f23096f9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23040b4c40 con 0x7f230415deb0 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23040cf360 0x7f23040b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23040cf360 0x7f23040b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42266/0 (socket says 172.21.15.114:42266) 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 -- 172.21.15.114:0/2950812403 learned_addr learned my addr 172.21.15.114:0/2950812403 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23037fe700 1 --2- 172.21.15.114:0/2950812403 >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 0x7f230415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 -- 172.21.15.114:0/2950812403 >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 msgr2=0x7f230415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 --2- 172.21.15.114:0/2950812403 >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 0x7f230415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 -- 172.21.15.114:0/2950812403 >> v1:172.21.15.172:6789/0 conn(0x7f230415a3b0 legacy=0x7f23040b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 -- 172.21.15.114:0/2950812403 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304070c60 con 0x7f23040cf360 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23037fe700 1 --2- 172.21.15.114:0/2950812403 >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 0x7f230415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f23027fc700 1 --2- 172.21.15.114:0/2950812403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23040cf360 0x7f23040b3a30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2304074fd0 tx=0x7f22f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f02c4a281756a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f2301ffb700 1 -- 172.21.15.114:0/2950812403 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400b870 con 0x7f23040cf360 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f2301ffb700 1 -- 172.21.15.114:0/2950812403 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400b9d0 con 0x7f23040cf360 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.792+0000 7f2301ffb700 1 -- 172.21.15.114:0/2950812403 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400bd60 con 0x7f23040cf360 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 -- 172.21.15.114:0/2950812403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23040cf360 msgr2=0x7f23040b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 --2- 172.21.15.114:0/2950812403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23040cf360 0x7f23040b3a30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2304074fd0 tx=0x7f22f4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 -- 172.21.15.114:0/2950812403 shutdown_connections 2023-11-24T06:27:54.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 --2- 172.21.15.114:0/2950812403 >> v2:172.21.15.172:3300/0 conn(0x7f230415deb0 0x7f230415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 --2- 172.21.15.114:0/2950812403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23040cf360 0x7f23040b3a30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 -- 172.21.15.114:0/2950812403 >> 172.21.15.114:0/2950812403 conn(0x7f23040b1740 msgr2=0x7f23040b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 -- 172.21.15.114:0/2950812403 shutdown_connections 2023-11-24T06:27:54.799 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.793+0000 7f23096f9700 1 -- 172.21.15.114:0/2950812403 wait complete. 2023-11-24T06:27:54.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.794+0000 7f23096f9700 1 Processor -- start 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.794+0000 7f23096f9700 1 -- start start 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23040cf360 0x7f23041f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 0x7f23041f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041f4ef0 con 0x7f230415a3b0 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041f4930 con 0x7f230415deb0 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23096f9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23041f4c10 con 0x7f23040cf360 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42272/0 (socket says 172.21.15.114:42272) 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 -- 172.21.15.114:0/465312421 learned_addr learned my addr 172.21.15.114:0/465312421 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23037fe700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 0x7f23041f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f2302ffd700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23040cf360 0x7f23041f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 -- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23040cf360 msgr2=0x7f23041f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23040cf360 0x7f23041f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 -- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 msgr2=0x7f23041f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 0x7f23041f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 -- 172.21.15.114:0/465312421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4009550 con 0x7f230415a3b0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23037fe700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 0x7f23041f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.795+0000 7f23027fc700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f22f4000c00 tx=0x7f22f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.796+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4004520 con 0x7f230415a3b0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.796+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f401e070 con 0x7f230415a3b0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.796+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2304152090 con 0x7f230415a3b0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.796+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40038a0 con 0x7f230415a3b0 2023-11-24T06:27:54.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.796+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23041f55c0 con 0x7f230415a3b0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.797+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2304151ef0 con 0x7f230415a3b0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.797+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f22f4003a00 con 0x7f230415a3b0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.797+0000 7f22eb7fe700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 0x7f22f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.797+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22f4020070 con 0x7f230415a3b0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.797+0000 7f2302ffd700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 0x7f22f006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.798+0000 7f2302ffd700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 0x7f22f006d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2304153cf0 tx=0x7f22f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:54.802+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2304151ef0 con 0x7f230415a3b0 2023-11-24T06:27:55.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.037+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2304151ef0 con 0x7f230415a3b0 2023-11-24T06:27:55.040 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.038+0000 7f22eb7fe700 1 -- 172.21.15.114:0/465312421 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f2304151ef0 con 0x7f230415a3b0 2023-11-24T06:27:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 msgr2=0x7f22f006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 0x7f22f006d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2304153cf0 tx=0x7f22f8009040 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 msgr2=0x7f230414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f22f4000c00 tx=0x7f22f4004750 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 shutdown_connections 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22f006b080 0x7f22f006d540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230415deb0 0x7f23041f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23040cf360 0x7f23041f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 --2- 172.21.15.114:0/465312421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f230415a3b0 0x7f230414f6e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 >> 172.21.15.114:0/465312421 conn(0x7f23040b1740 msgr2=0x7f230406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 shutdown_connections 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.040+0000 7f23096f9700 1 -- 172.21.15.114:0/465312421 wait complete. 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2023-11-24T06:27:55.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:27:44.273957+0000', 'modified': '2023-11-24T06:27:47.590785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6216}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6216': {'gid': 6216, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6837/3988687339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6835', 'nonce': 3988687339}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 3988687339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:27:55.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:27:55.458 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.578+0000 7fcf7c52d700 1 Processor -- start 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.578+0000 7fcf7c52d700 1 -- start start 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.579+0000 7fcf7c52d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf740d2380 0x7fcf741604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.579+0000 7fcf7c52d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740b22d0 con 0x7fcf7415bdd0 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.579+0000 7fcf7c52d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740b25b0 con 0x7fcf740b3370 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.579+0000 7fcf7c52d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740b2f50 con 0x7fcf740d2380 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7aaca700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf740d2380 0x7fcf741604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7a2c9700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcf7415bdd0 0x7fcf740b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57986/0 (socket says 172.21.15.114:57986) 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/3323262380 learned_addr learned my addr 172.21.15.114:0/3323262380 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3323262380 >> v1:172.21.15.172:6789/0 conn(0x7fcf740b3370 legacy=0x7fcf740d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3323262380 >> v1:172.21.15.114:6789/0 conn(0x7fcf7415bdd0 legacy=0x7fcf740b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3323262380 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740b2890 con 0x7fcf740d2380 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf792c7700 1 -- 172.21.15.114:0/3323262380 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290595607 0 0) 0x7fcf740b22d0 con 0x7fcf7415bdd0 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3323262380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740d2380 0x7fcf741604e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009d90 tx=0x7fcf6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1913237c0ab11137 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf792c7700 1 -- 172.21.15.114:0/3323262380 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e070 con 0x7fcf740d2380 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf792c7700 1 -- 172.21.15.114:0/3323262380 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00bb90 con 0x7fcf740d2380 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.580+0000 7fcf792c7700 1 -- 172.21.15.114:0/3323262380 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c003a20 con 0x7fcf740d2380 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 -- 172.21.15.114:0/3323262380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740d2380 msgr2=0x7fcf741604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/3323262380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740d2380 0x7fcf741604e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009d90 tx=0x7fcf6c00b330 comp rx=0 tx=0).stop 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 -- 172.21.15.114:0/3323262380 shutdown_connections 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/3323262380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740d2380 0x7fcf741604e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.584 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 -- 172.21.15.114:0/3323262380 >> 172.21.15.114:0/3323262380 conn(0x7fcf740731a0 msgr2=0x7fcf7409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 -- 172.21.15.114:0/3323262380 shutdown_connections 2023-11-24T06:27:55.585 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.581+0000 7fcf7c52d700 1 -- 172.21.15.114:0/3323262380 wait complete. 2023-11-24T06:27:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.582+0000 7fcf7c52d700 1 Processor -- start 2023-11-24T06:27:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.582+0000 7fcf7c52d700 1 -- start start 2023-11-24T06:27:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 0x7fcf7415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 0x7fcf74152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b4b40 con 0x7fcf740b3370 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b4d50 con 0x7fcf7415bdd0 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b5330 con 0x7fcf740d2380 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42284/0 (socket says 172.21.15.114:42284) 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/2767779563 learned_addr learned my addr 172.21.15.114:0/2767779563 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 0x7fcf74152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 0x7fcf7415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 msgr2=0x7fcf7415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 0x7fcf7415f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 msgr2=0x7fcf74152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 0x7fcf74152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/2767779563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009a70 con 0x7fcf740b3370 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 0x7fcf74152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 0x7fcf7415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c550 tx=0x7fcf6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf6affd700 1 -- 172.21.15.114:0/2767779563 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64004c80 con 0x7fcf740b3370 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf6affd700 1 -- 172.21.15.114:0/2767779563 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64004e00 con 0x7fcf740b3370 2023-11-24T06:27:55.589 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740b5610 con 0x7fcf740b3370 2023-11-24T06:27:55.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf6affd700 1 -- 172.21.15.114:0/2767779563 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64007310 con 0x7fcf740b3370 2023-11-24T06:27:55.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.583+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741534f0 con 0x7fcf740b3370 2023-11-24T06:27:55.590 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.585+0000 7fcf6affd700 1 -- 172.21.15.114:0/2767779563 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcf64007470 con 0x7fcf740b3370 2023-11-24T06:27:55.592 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.585+0000 7fcf6affd700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 0x7fcf5806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.586+0000 7fcf6affd700 1 -- 172.21.15.114:0/2767779563 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf64089620 con 0x7fcf740b3370 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.586+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 0x7fcf5806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.586+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 0x7fcf5806d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf74153a10 tx=0x7fcf70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.587+0000 7fcf88726b80 1 Processor -- start 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.587+0000 7fcf88726b80 1 -- start start 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x5650471cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565047115770 con 0x5650471e7ac0 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650470fc750 con 0x5650471cc060 2023-11-24T06:27:55.593 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650470fa070 con 0x5650471e8ab0 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf7aaca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf7aaca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42296/0 (socket says 172.21.15.114:42296) 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3821193905 learned_addr learned my addr 172.21.15.114:0/3821193905 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.588+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x5650471cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 msgr2=0x5650471e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471e7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 msgr2=0x5650471cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x5650471cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 -- 172.21.15.114:0/3821193905 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565046f725e0 con 0x5650471e7ac0 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:55.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcf740716d0 tx=0x7fcf6c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9445c5d891e1c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf57fff700 1 -- 172.21.15.114:0/3821193905 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004030 con 0x5650471e7ac0 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf57fff700 1 -- 172.21.15.114:0/3821193905 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c004190 con 0x5650471e7ac0 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf57fff700 1 -- 172.21.15.114:0/3821193905 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004500 con 0x5650471e7ac0 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf88726b80 1 -- 172.21.15.114:0/3821193905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 msgr2=0x5650471e4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.589+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcf740716d0 tx=0x7fcf6c010000 comp rx=0 tx=0).stop 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 -- 172.21.15.114:0/3821193905 shutdown_connections 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471e4e00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x5650471cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3821193905 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 -- 172.21.15.114:0/3821193905 >> 172.21.15.114:0/3821193905 conn(0x5650470d0310 msgr2=0x5650471e6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 -- 172.21.15.114:0/3821193905 shutdown_connections 2023-11-24T06:27:55.595 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 -- 172.21.15.114:0/3821193905 wait complete. 2023-11-24T06:27:55.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 -- start start 2023-11-24T06:27:55.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.590+0000 7fcf88726b80 1 Processor -- start 2023-11-24T06:27:55.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471cc060 0x5650471e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 0x5650471b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650470f5960 con 0x5650471e7ac0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650470af510 con 0x5650471e8ab0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650470af1b0 con 0x5650471cc060 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf88726b80 10 client.0 _ll_register_callbacks cb 0x565046fdaa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf79ac8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf79ac8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42300/0 (socket says 172.21.15.114:42300) 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf79ac8700 1 -- 172.21.15.114:0/3725456571 learned_addr learned my addr 172.21.15.114:0/3725456571 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471cc060 0x5650471e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.591+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 0x5650471b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 -- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471cc060 msgr2=0x5650471e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471cc060 0x5650471e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 -- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 msgr2=0x5650471b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 0x5650471b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 -- 172.21.15.114:0/3725456571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740b2890 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 0x5650471b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf70003d70 tx=0x7fcf70000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd750ae4a86c2b51 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70014bd0 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf70014d30 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565047194a40 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf70021430 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 10 client.6366 fetch_fsmap learned FSMap version 37 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x565046f1ed90 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565047194a40 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf70021590 con 0x5650471e7ac0 2023-11-24T06:27:55.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 10 client.6366 fetch_fsmap finished waiting for FSMap version 37 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 10 client.6366 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 20 client.6366 populate_metadata read hostname 'smithi114' 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471a0190 0x5650471a0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 4 client.6366 mds_command: new command op to 6216 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 --> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565047170840 con 0x5650471a0190 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.592+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471a0190 0x5650471a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.593+0000 7fcf557fa700 10 client.6366 ms_handle_connect on v2:172.21.15.114:6835/3988687339 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.593+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471a0190 0x5650471a0570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1dfe3dcfc5a5cd server_cookie=a91adc09a05f31e5 in_seq=0 out_seq=0 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.593+0000 7fcf557fa700 1 -- 172.21.15.114:0/3725456571 <== mds.0 v2:172.21.15.114:6835/3988687339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565047170840 con 0x5650471a0190 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.593+0000 7fcf557fa700 10 client.6366 handle_command_reply: tid=0 2023-11-24T06:27:55.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.593+0000 7fcf88726b80 1 client.6366 shutdown 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 msgr2=0x5650471cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcf70003d70 tx=0x7fcf70000fb0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 shutdown_connections 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e8ab0 0x5650471b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471cc060 0x5650471e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471a0190 0x5650471a0570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 --2- 172.21.15.114:0/3725456571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e7ac0 0x5650471cd620 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf557fa700 0 client.6366 ms_handle_reset on v2:172.21.15.114:6835/3988687339 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 >> 172.21.15.114:0/3725456571 conn(0x5650470d0310 msgr2=0x565047196330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 shutdown_connections 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 1 -- 172.21.15.114:0/3725456571 wait complete. 2023-11-24T06:27:55.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.594+0000 7fcf88726b80 20 client.6366 trim_cache size 0 max 16384 2023-11-24T06:27:55.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 Processor -- start 2023-11-24T06:27:55.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 -- start start 2023-11-24T06:27:55.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.616 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e7ac0 0x5650471dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565046f748d0 con 0x5650471cc060 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650471e4540 con 0x5650471e7ac0 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.612+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650471cd0b0 con 0x5650471e8ab0 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42306/0 (socket says 172.21.15.114:42306) 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/783448052 learned_addr learned my addr 172.21.15.114:0/783448052 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e7ac0 0x5650471dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 msgr2=0x5650471cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e7ac0 msgr2=0x5650471dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e7ac0 0x5650471dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 -- 172.21.15.114:0/783448052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565046f07c10 con 0x5650471cc060 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c520 tx=0x7fcf6405cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced7b885adcb0bbf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf68ff9700 1 -- 172.21.15.114:0/783448052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64058100 con 0x5650471cc060 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf68ff9700 1 -- 172.21.15.114:0/783448052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6405c250 con 0x5650471cc060 2023-11-24T06:27:55.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.613+0000 7fcf68ff9700 1 -- 172.21.15.114:0/783448052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64060400 con 0x5650471cc060 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- 172.21.15.114:0/783448052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 msgr2=0x56504719d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c520 tx=0x7fcf6405cf90 comp rx=0 tx=0).stop 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- 172.21.15.114:0/783448052 shutdown_connections 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e8ab0 0x5650471cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471cc060 0x56504719d790 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 --2- 172.21.15.114:0/783448052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471e7ac0 0x5650471dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- 172.21.15.114:0/783448052 >> 172.21.15.114:0/783448052 conn(0x5650470d0310 msgr2=0x5650471e7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- 172.21.15.114:0/783448052 shutdown_connections 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- 172.21.15.114:0/783448052 wait complete. 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 -- start start 2023-11-24T06:27:55.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.614+0000 7fcf88726b80 1 Processor -- start 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x565047196370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e7ac0 0x565047203e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504719f060 con 0x5650471e8ab0 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650471968b0 con 0x5650471cc060 2023-11-24T06:27:55.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650471d0010 con 0x5650471e7ac0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf88726b80 10 client.0 _ll_register_callbacks cb 0x565046fdaa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf7aaca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf7aaca700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42316/0 (socket says 172.21.15.114:42316) 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf7aaca700 1 -- 172.21.15.114:0/4196361828 learned_addr learned my addr 172.21.15.114:0/4196361828 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf79ac8700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e7ac0 0x565047203e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.615+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x565047196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 -- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e7ac0 msgr2=0x565047203e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e7ac0 0x565047203e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 -- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 msgr2=0x565047196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x565047196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 -- 172.21.15.114:0/4196361828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741534f0 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7aaca700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f80 tx=0x7fcf6c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6002af7920cd01a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c031640 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf6c004190 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6c02b040 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6c00b7a0 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 10 client.6372 fetch_fsmap learned FSMap version 37 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x565046f45200 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c004190 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf6c024030 con 0x5650471e8ab0 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 10 client.6372 fetch_fsmap finished waiting for FSMap version 37 2023-11-24T06:27:55.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 10 client.6372 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 20 client.6372 populate_metadata read hostname 'smithi114' 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471cf2c0 0x5650471cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 4 client.6372 mds_command: new command op to 6216 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 --> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650470bbf80 con 0x5650471cf2c0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.616+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471cf2c0 0x5650471cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf557fa700 10 client.6372 ms_handle_connect on v2:172.21.15.114:6835/3988687339 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf7a2c9700 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471cf2c0 0x5650471cf6a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323761385ac2baa4 server_cookie=c10ae737e17b930b in_seq=0 out_seq=0 2023-11-24T06:27:55.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf557fa700 1 -- 172.21.15.114:0/4196361828 <== mds.0 v2:172.21.15.114:6835/3988687339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5650470bbf80 con 0x5650471cf2c0 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf557fa700 10 client.6372 handle_command_reply: tid=0 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 client.6372 shutdown 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 msgr2=0x565047204360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f80 tx=0x7fcf6c004b40 comp rx=0 tx=0).stop 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 shutdown_connections 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:6835/3988687339,v1:172.21.15.114:6837/3988687339] conn(0x5650471cf2c0 0x5650471cf6a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650471e8ab0 0x565047204360 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5650471cc060 0x565047196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 --2- 172.21.15.114:0/4196361828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5650471e7ac0 0x565047203e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 >> 172.21.15.114:0/4196361828 conn(0x5650470d0310 msgr2=0x5650471cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf557fa700 0 client.6372 ms_handle_reset on v2:172.21.15.114:6835/3988687339 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 shutdown_connections 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 1 -- 172.21.15.114:0/4196361828 wait complete. 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.617+0000 7fcf88726b80 20 client.6372 trim_cache size 0 max 16384 2023-11-24T06:27:55.632 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "cfb0acb2-02f2-43f9-bba4-c35657dcf6fd", 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 6216, 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 10.436960328, 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 37, 2023-11-24T06:27:55.633 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 58, 2023-11-24T06:27:55.634 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 58, 2023-11-24T06:27:55.634 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 16.370425017999999 2023-11-24T06:27:55.634 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 msgr2=0x7fcf5806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 0x7fcf5806d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf74153a10 tx=0x7fcf70009040 comp rx=0 tx=0).stop 2023-11-24T06:27:55.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 msgr2=0x7fcf740cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c550 tx=0x7fcf6400c910 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 shutdown_connections 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fcf5806b080 0x7fcf5806d540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf7415bdd0 0x7fcf74152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf740b3370 0x7fcf740cbe40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 --2- 172.21.15.114:0/2767779563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf740d2380 0x7fcf7415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 >> 172.21.15.114:0/2767779563 conn(0x7fcf740731a0 msgr2=0x7fcf7406ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 shutdown_connections 2023-11-24T06:27:55.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.618+0000 7fcf7c52d700 1 -- 172.21.15.114:0/2767779563 wait complete. 2023-11-24T06:27:55.636 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:55.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.754+0000 7f88f92fc700 1 Processor -- start 2023-11-24T06:27:55.757 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.754+0000 7f88f92fc700 1 -- start start 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f92fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f88f406c710 0x7f88f40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f92fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 0x7f88f40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f92fc700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40ae430 con 0x7f88f415e080 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f92fc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40af0b0 con 0x7f88f406c710 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f92fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40b05a0 con 0x7f88f40af4d0 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f27fc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f88f406c710 0x7f88f40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f2ffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f88f415e080 0x7f88f406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58028/0 (socket says 172.21.15.114:58028) 2023-11-24T06:27:55.759 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.755+0000 7f88f2ffd700 1 -- 172.21.15.114:0/2074388470 learned_addr learned my addr 172.21.15.114:0/2074388470 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f37fe700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 0x7f88f40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f27fc700 1 -- 172.21.15.114:0/2074388470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 msgr2=0x7f88f40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f27fc700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 0x7f88f40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f27fc700 1 -- 172.21.15.114:0/2074388470 >> v1:172.21.15.114:6789/0 conn(0x7f88f415e080 legacy=0x7f88f406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f27fc700 1 -- 172.21.15.114:0/2074388470 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40ae710 con 0x7f88f406c710 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f1ffb700 1 -- 172.21.15.114:0/2074388470 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284674261 0 0) 0x7f88f40ae430 con 0x7f88f415e080 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f37fe700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 0x7f88f40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f27fc700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f406c710 0x7f88f40b5050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009840 tx=0x7f88e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2758016369ff9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f1ffb700 1 -- 172.21.15.114:0/2074388470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8011830 con 0x7f88f406c710 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f1ffb700 1 -- 172.21.15.114:0/2074388470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8011990 con 0x7f88f406c710 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f1ffb700 1 -- 172.21.15.114:0/2074388470 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8011d20 con 0x7f88f406c710 2023-11-24T06:27:55.760 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 -- 172.21.15.114:0/2074388470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f406c710 msgr2=0x7f88f40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f406c710 0x7f88f40b5050 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009840 tx=0x7f88e8009b50 comp rx=0 tx=0).stop 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 -- 172.21.15.114:0/2074388470 shutdown_connections 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f40af4d0 0x7f88f40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 --2- 172.21.15.114:0/2074388470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f406c710 0x7f88f40b5050 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.756+0000 7f88f92fc700 1 -- 172.21.15.114:0/2074388470 >> 172.21.15.114:0/2074388470 conn(0x7f88f4071b90 msgr2=0x7f88f4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.757+0000 7f88f92fc700 1 -- 172.21.15.114:0/2074388470 shutdown_connections 2023-11-24T06:27:55.761 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.757+0000 7f88f92fc700 1 -- 172.21.15.114:0/2074388470 wait complete. 2023-11-24T06:27:55.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.758+0000 7f88f92fc700 1 Processor -- start 2023-11-24T06:27:55.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.758+0000 7f88f92fc700 1 -- start start 2023-11-24T06:27:55.763 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f406c710 0x7f88f4158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f40af4d0 0x7f88f40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40d0620 con 0x7f88f415e080 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f41586b0 con 0x7f88f40af4d0 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4158990 con 0x7f88f406c710 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42330/0 (socket says 172.21.15.114:42330) 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 -- 172.21.15.114:0/4044258801 learned_addr learned my addr 172.21.15.114:0/4044258801 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f2ffd700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f406c710 0x7f88f4158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f27fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f40af4d0 0x7f88f40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 -- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f406c710 msgr2=0x7f88f4158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.764 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f406c710 0x7f88f4158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 -- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f40af4d0 msgr2=0x7f88f40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f40af4d0 0x7f88f40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 -- 172.21.15.114:0/4044258801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80094a0 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f37fe700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00cef0 tx=0x7f88ec00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00ad00 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec00ae60 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f4158c70 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec015680 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.759+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f40cd900 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.760+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f40753f0 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.761+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f88ec00d430 con 0x7f88f415e080 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.761+0000 7f88db7fe700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 0x7f88e006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.765 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.761+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88ec089dc0 con 0x7f88f415e080 2023-11-24T06:27:55.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.761+0000 7f88f2ffd700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 0x7f88e006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.766 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.762+0000 7f88f2ffd700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 0x7f88e006d540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88e4000f60 tx=0x7f88e4007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.768 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:55.766+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88ec054bf0 con 0x7f88f415e080 2023-11-24T06:27:56.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.002+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f88f4067f60 con 0x7f88f415e080 2023-11-24T06:27:56.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.002+0000 7f88db7fe700 1 -- 172.21.15.114:0/4044258801 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f88ec058210 con 0x7f88f415e080 2023-11-24T06:27:56.005 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 msgr2=0x7f88e006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 0x7f88e006d540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88e4000f60 tx=0x7f88e4007e70 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 msgr2=0x7f88f414a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00cef0 tx=0x7f88ec00cf20 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 shutdown_connections 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f88e006b080 0x7f88e006d540 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f415e080 0x7f88f414a3d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88f406c710 0x7f88f4158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 --2- 172.21.15.114:0/4044258801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88f40af4d0 0x7f88f40d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.007 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 >> 172.21.15.114:0/4044258801 conn(0x7f88f4071b90 msgr2=0x7f88f40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 shutdown_connections 2023-11-24T06:27:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.004+0000 7f88f92fc700 1 -- 172.21.15.114:0/4044258801 wait complete. 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:56.019 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:27:56.020 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:27:56.020 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:56.020 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:27:56.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.020 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2023-11-24T06:27:56.038 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:56.038 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:56.039 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:27:56.040 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42179sec preferred_lft 42179sec 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::9ca9:e760:b5c4:9eac/64 scope link noprefixroute 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::385c:94ff:fec5:c8f4/64 scope link 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether fa:2a:57:62:51:8a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::f82a:57ff:fe62:518a/64 scope link 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:56.042 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether ae:f4:a2:10:15:ae brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:56.043 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::acf4:a2ff:fe10:15ae/64 scope link 2023-11-24T06:27:56.043 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:56.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.043 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:56.105 INFO:teuthology.orchestra.run.smithi114.stdout:1 2023-11-24T06:27:56.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.105 DEBUG:teuthology.orchestra.run.smithi114:> route 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:56.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.164 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2023-11-24T06:27:56.164 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2023-11-24T06:27:56.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:27:56.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:27:56.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:27:56.165 DEBUG:teuthology.orchestra.run.smithi114:> ') 2023-11-24T06:27:56.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.270 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:27:56.287 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:27:56.287 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:27:56.287 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:27:56.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2023-11-24T06:27:56.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:56.354 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:56.413 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:27:56.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:56.413 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:56.470 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:27:56.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:56.556 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:56.556 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:56.583 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:27:56.584 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:27:56.662 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with new device syntax: 0@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:27:56.663 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:27:56.664 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:27:56.664 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:27:56.664 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:27:56.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:56.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:56.732 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:27:56.817 INFO:teuthology.orchestra.run.smithi114.stdout:client5728 2023-11-24T06:27:56.818 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:56.818 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:56.835 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.993+0000 7f0f971e4700 1 Processor -- start 2023-11-24T06:27:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.993+0000 7f0f971e4700 1 -- start start 2023-11-24T06:27:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.993+0000 7f0f971e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9006c710 0x7f0f900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f971e4700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0f900af4d0 0x7f0f900afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f971e4700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900ae430 con 0x7f0f9015e080 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f971e4700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f900af0b0 con 0x7f0f900af4d0 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f971e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b05a0 con 0x7f0f9006c710 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f94f80700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0f9015e080 0x7f0f9006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58046/0 (socket says 172.21.15.114:58046) 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f94f80700 1 -- 172.21.15.114:0/162929658 learned_addr learned my addr 172.21.15.114:0/162929658 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9006c710 0x7f0f900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f95781700 1 --2- 172.21.15.114:0/162929658 >> v2:172.21.15.172:3300/0 conn(0x7f0f900af4d0 0x7f0f900afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463577692 0 0) 0x7f0f900ae430 con 0x7f0f9015e080 2023-11-24T06:27:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f800035b0 con 0x7f0f9015e080 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274650938 0 0) 0x7f0f800035b0 con 0x7f0f9015e080 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9006c710 msgr2=0x7f0f900b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9006c710 0x7f0f900b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 >> v2:172.21.15.172:3300/0 conn(0x7f0f900af4d0 msgr2=0x7f0f900afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 --2- 172.21.15.114:0/162929658 >> v2:172.21.15.172:3300/0 conn(0x7f0f900af4d0 0x7f0f900afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900ae710 con 0x7f0f9015e080 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0f84001670 con 0x7f0f9015e080 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 >> v1:172.21.15.114:6789/0 conn(0x7f0f9015e080 legacy=0x7f0f9006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f80003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f80004850 0x7f0f80004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f80005210 0x7f0f8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900af0b0 con 0x7f0f80003a30 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900b05a0 con 0x7f0f80004850 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8000bf50 con 0x7f0f80005210 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f9006c710 0x7f0f900b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f95781700 1 --2- 172.21.15.114:0/162929658 >> v2:172.21.15.172:3300/0 conn(0x7f0f900af4d0 0x7f0f900afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.994+0000 7f0f94f80700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f80003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f95781700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f80005210 0x7f0f8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f80004850 0x7f0f80004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 -- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f80005210 msgr2=0x7f0f8000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f80005210 0x7f0f8000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 -- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f80004850 msgr2=0x7f0f80004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f80004850 0x7f0f80004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 -- 172.21.15.114:0/162929658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800035b0 con 0x7f0f80003a30 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f94f80700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f80003e50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c240 tx=0x7f0f80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3318159c98060c72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84007c10 con 0x7f0f80003a30 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8400cda0 con 0x7f0f80003a30 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.995+0000 7f0f8f7fe700 1 -- 172.21.15.114:0/162929658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84005ba0 con 0x7f0f80003a30 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 -- 172.21.15.114:0/162929658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 msgr2=0x7f0f80003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 --2- 172.21.15.114:0/162929658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f80003e50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c240 tx=0x7f0f80005880 comp rx=0 tx=0).stop 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f95781700 1 -- 172.21.15.114:0/162929658 reap_dead start 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 -- 172.21.15.114:0/162929658 shutdown_connections 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 -- 172.21.15.114:0/162929658 >> 172.21.15.114:0/162929658 conn(0x7f0f90071b90 msgr2=0x7f0f90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 -- 172.21.15.114:0/162929658 shutdown_connections 2023-11-24T06:27:57.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.996+0000 7f0f971e4700 1 -- 172.21.15.114:0/162929658 wait complete. 2023-11-24T06:27:57.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.997+0000 7f0f971e4700 1 Processor -- start 2023-11-24T06:27:57.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.997+0000 7f0f971e4700 1 -- start start 2023-11-24T06:27:57.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 0x7f0f901595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f90157250 0x7f0f901576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f90157c10 con 0x7f0f80003a30 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901424e0 con 0x7f0f90159120 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f971e4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f90159c70 con 0x7f0f90157250 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f94f80700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f94f80700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42350/0 (socket says 172.21.15.114:42350) 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.998+0000 7f0f94f80700 1 -- 172.21.15.114:0/321568644 learned_addr learned my addr 172.21.15.114:0/321568644 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f95781700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f90157250 0x7f0f901576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 0x7f0f901595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 -- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f90157250 msgr2=0x7f0f901576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f90157250 0x7f0f901576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 -- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 msgr2=0x7f0f901595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 0x7f0f901595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 -- 172.21.15.114:0/321568644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90159f50 con 0x7f0f80003a30 2023-11-24T06:27:57.012 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 0x7f0f901595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f94f80700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c830 tx=0x7f0f84001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84004d40 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f840070d0 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9015a230 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84004ea0 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:56.999+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9015f080 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.000+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f900753f0 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.000+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0f84019070 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.001+0000 7f0f8d7fa700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 0x7f0f7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.001+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f8408a8c0 con 0x7f0f80003a30 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.001+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 0x7f0f7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.013 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.001+0000 7f0f8ffff700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 0x7f0f7406d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f901f7430 tx=0x7f0f7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.014 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.005+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f84055780 con 0x7f0f80003a30 2023-11-24T06:27:57.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.241+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f90067f60 con 0x7f0f80003a30 2023-11-24T06:27:57.244 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.242+0000 7f0f8d7fa700 1 -- 172.21.15.114:0/321568644 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f84017060 con 0x7f0f80003a30 2023-11-24T06:27:57.244 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 msgr2=0x7f0f7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 0x7f0f7406d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f901f7430 tx=0x7f0f7c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 msgr2=0x7f0f90141fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c830 tx=0x7f0f84001a00 comp rx=0 tx=0).stop 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 shutdown_connections 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f7406b080 0x7f0f7406d540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f90157250 0x7f0f901576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f80003a30 0x7f0f90141fa0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 --2- 172.21.15.114:0/321568644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f90159120 0x7f0f901595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 >> 172.21.15.114:0/321568644 conn(0x7f0f90071b90 msgr2=0x7f0f900b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 shutdown_connections 2023-11-24T06:27:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:57.244+0000 7f0f971e4700 1 -- 172.21.15.114:0/321568644 wait complete. 2023-11-24T06:27:57.258 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:27:57.258 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:57.258 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:57.259 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:27:57.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.259 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:57.281 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42202sec preferred_lft 42202sec 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::c36d:1855:677d:d218/64 scope link noprefixroute 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:27:57.282 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::4cb3:adff:fe77:17e6/64 scope link 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 3a:38:94:a3:16:87 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::3838:94ff:fea3:1687/64 scope link 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 56:7e:24:58:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::547e:24ff:fe58:9cd/64 scope link 2023-11-24T06:27:57.283 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:57.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.284 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:57.351 INFO:teuthology.orchestra.run.smithi172.stdout:1 2023-11-24T06:27:57.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.353 DEBUG:teuthology.orchestra.run.smithi172:> route 2023-11-24T06:27:57.415 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2023-11-24T06:27:57.416 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:57.416 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:27:57.416 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:27:57.416 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:57.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.416 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2023-11-24T06:27:57.416 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2023-11-24T06:27:57.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:27:57.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:27:57.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:27:57.417 DEBUG:teuthology.orchestra.run.smithi172:> ') 2023-11-24T06:27:57.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.526 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:27:57.543 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:27:57.543 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:27:57.543 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:27:57.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.544 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2023-11-24T06:27:57.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:57.610 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:57.667 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:27:57.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:57.667 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:57.724 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:27:57.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:57.809 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:57.809 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:57.835 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:27:57.836 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.172:3300,172.21.15.172:3301" 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with new device syntax: 1@cfb0acb2-02f2-43f9-bba4-c35657dcf6fd.cephfs=/ 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.114:3300/172.21.15.172:3300/172.21.15.172:3301" will pass to kernel 2023-11-24T06:27:57.866 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:27:57.867 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:27:57.867 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:27:57.867 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790" 2023-11-24T06:27:57.867 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2023-11-24T06:27:57.867 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=cfb0acb2-02f2-43f9-bba4-c35657dcf6fd" will pass to kernel 2023-11-24T06:27:57.869 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:27:57.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:57.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.900 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:27:57.973 INFO:teuthology.orchestra.run.smithi172.stdout:client5734 2023-11-24T06:27:57.973 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:27:57.973 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:58.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:27:58.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.162+0000 7f8ce58f0700 1 Processor -- start 2023-11-24T06:27:58.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.162+0000 7f8ce58f0700 1 -- start start 2023-11-24T06:27:58.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8ce58f0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8ce015e120 0x7f8ce006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8ce58f0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce00af5e0 con 0x7f8ce00af9b0 2023-11-24T06:27:58.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8ce58f0700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce006dd80 con 0x7f8ce00b4260 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8ce58f0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce00b0350 con 0x7f8ce015e120 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8ce015e120 0x7f8ce006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8ce015e120 0x7f8ce006eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42358/0 (socket says 172.21.15.114:42358) 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 -- 172.21.15.114:0/902579540 learned_addr learned my addr 172.21.15.114:0/902579540 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 -- 172.21.15.114:0/902579540 >> v1:172.21.15.172:6789/0 conn(0x7f8ce00b4260 legacy=0x7f8ce006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 -- 172.21.15.114:0/902579540 >> v1:172.21.15.114:6789/0 conn(0x7f8ce00af9b0 legacy=0x7f8ce00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 -- 172.21.15.114:0/902579540 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce006e040 con 0x7f8ce015e120 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cddffb700 1 -- 172.21.15.114:0/902579540 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765878150 0 0) 0x7f8ce00af5e0 con 0x7f8ce00af9b0 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cdeffd700 1 --2- 172.21.15.114:0/902579540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce006eb20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00098f0 tx=0x7f8cd0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e33f597f1cda76 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cddffb700 1 -- 172.21.15.114:0/902579540 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0004230 con 0x7f8ce015e120 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cddffb700 1 -- 172.21.15.114:0/902579540 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd0004390 con 0x7f8ce015e120 2023-11-24T06:27:58.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.163+0000 7f8cddffb700 1 -- 172.21.15.114:0/902579540 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00036a0 con 0x7f8ce015e120 2023-11-24T06:27:58.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 -- 172.21.15.114:0/902579540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 msgr2=0x7f8ce006eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/902579540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce006eb20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00098f0 tx=0x7f8cd0009c00 comp rx=0 tx=0).stop 2023-11-24T06:27:58.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 -- 172.21.15.114:0/902579540 shutdown_connections 2023-11-24T06:27:58.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/902579540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce006eb20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 -- 172.21.15.114:0/902579540 >> 172.21.15.114:0/902579540 conn(0x7f8ce00d0ba0 msgr2=0x7f8ce00d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 -- 172.21.15.114:0/902579540 shutdown_connections 2023-11-24T06:27:58.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.164+0000 7f8ce58f0700 1 -- 172.21.15.114:0/902579540 wait complete. 2023-11-24T06:27:58.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.165+0000 7f8ce58f0700 1 Processor -- start 2023-11-24T06:27:58.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.165+0000 7f8ce58f0700 1 -- start start 2023-11-24T06:27:58.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ce00af9b0 0x7f8ce00adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ce00b4260 0x7f8ce01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce0158f30 con 0x7f8ce015e120 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce015f360 con 0x7f8ce00b4260 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8ce58f0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce015e5a0 con 0x7f8ce00af9b0 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42368/0 (socket says 172.21.15.114:42368) 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 -- 172.21.15.114:0/3376470632 learned_addr learned my addr 172.21.15.114:0/3376470632 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdeffd700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ce00af9b0 0x7f8ce00adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 -- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ce00af9b0 msgr2=0x7f8ce00adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ce00af9b0 0x7f8ce00adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 -- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ce00b4260 msgr2=0x7f8ce01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ce00b4260 0x7f8ce01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.166+0000 7f8cdf7fe700 1 -- 172.21.15.114:0/3376470632 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0009550 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8cdf7fe700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800cc10 tx=0x7f8cd800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800abb0 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd800ad10 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8012660 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce015e880 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.167+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce015ee40 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.168+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce0068480 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.168+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8cd80127c0 con 0x7f8ce015e120 2023-11-24T06:27:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.168+0000 7f8cc7fff700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 0x7f8ccc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.169+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cd808a0d0 con 0x7f8ce015e120 2023-11-24T06:27:58.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.169+0000 7f8cdeffd700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 0x7f8ccc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.169+0000 7f8cdeffd700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 0x7f8ccc06d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009130 tx=0x7f8cd0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.173+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8cd8054f00 con 0x7f8ce015e120 2023-11-24T06:27:58.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.408+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ce00751e0 con 0x7f8ce015e120 2023-11-24T06:27:58.886 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.884+0000 7f8cc7fff700 1 -- 172.21.15.114:0/3376470632 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8cd8058520 con 0x7f8ce015e120 2023-11-24T06:27:58.890 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 msgr2=0x7f8ccc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 0x7f8ccc06d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009130 tx=0x7f8cd0008030 comp rx=0 tx=0).stop 2023-11-24T06:27:58.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 msgr2=0x7f8ce01589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800cc10 tx=0x7f8cd800a060 comp rx=0 tx=0).stop 2023-11-24T06:27:58.891 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 shutdown_connections 2023-11-24T06:27:58.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8ccc06b080 0x7f8ccc06d540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ce015e120 0x7f8ce01589f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ce00af9b0 0x7f8ce00adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 --2- 172.21.15.114:0/3376470632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ce00b4260 0x7f8ce01584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.892 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 >> 172.21.15.114:0/3376470632 conn(0x7f8ce00d0ba0 msgr2=0x7f8ce00aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 shutdown_connections 2023-11-24T06:27:58.893 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:58.888+0000 7f8ce58f0700 1 -- 172.21.15.114:0/3376470632 wait complete. 2023-11-24T06:27:58.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:27:59.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.026+0000 7f0d6216d700 1 Processor -- start 2023-11-24T06:27:59.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.026+0000 7f0d6216d700 1 -- start start 2023-11-24T06:27:59.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d6216d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d5c15e080 0x7f0d5c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d6216d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d6216d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5c0ae430 con 0x7f0d5c06c000 2023-11-24T06:27:59.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d6216d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c0af0b0 con 0x7f0d5c15e080 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d6216d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0b05a0 con 0x7f0d5c0af4d0 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d5c15e080 0x7f0d5c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d5c15e080 0x7f0d5c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42372/0 (socket says 172.21.15.114:42372) 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/1236552915 learned_addr learned my addr 172.21.15.114:0/1236552915 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5affd700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/1236552915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 msgr2=0x7f0d5c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.027+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/1236552915 >> v1:172.21.15.172:6789/0 conn(0x7f0d5c06c000 legacy=0x7f0d5c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/1236552915 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c0ae710 con 0x7f0d5c15e080 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d5b7fe700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c15e080 0x7f0d5c0b5050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009920 tx=0x7f0d50009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29c2180068adc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d5a7fc700 1 -- 172.21.15.114:0/1236552915 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50004230 con 0x7f0d5c15e080 2023-11-24T06:27:59.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d5a7fc700 1 -- 172.21.15.114:0/1236552915 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50004390 con 0x7f0d5c15e080 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d5a7fc700 1 -- 172.21.15.114:0/1236552915 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50009e70 con 0x7f0d5c15e080 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 -- 172.21.15.114:0/1236552915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c15e080 msgr2=0x7f0d5c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c15e080 0x7f0d5c0b5050 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009920 tx=0x7f0d50009c30 comp rx=0 tx=0).stop 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 -- 172.21.15.114:0/1236552915 shutdown_connections 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c15e080 0x7f0d5c0b5050 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 --2- 172.21.15.114:0/1236552915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 -- 172.21.15.114:0/1236552915 >> 172.21.15.114:0/1236552915 conn(0x7f0d5c071b90 msgr2=0x7f0d5c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 -- 172.21.15.114:0/1236552915 shutdown_connections 2023-11-24T06:27:59.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.028+0000 7f0d6216d700 1 -- 172.21.15.114:0/1236552915 wait complete. 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 Processor -- start 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 -- start start 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 0x7f0d5c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c156f00 con 0x7f0d5c06c000 2023-11-24T06:27:59.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1571e0 con 0x7f0d5c15e080 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.030+0000 7f0d6216d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c15f4b0 con 0x7f0d5c0af4d0 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42378/0 (socket says 172.21.15.114:42378) 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/464179887 learned_addr learned my addr 172.21.15.114:0/464179887 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5affd700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5bfff700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 0x7f0d5c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 msgr2=0x7f0d5c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 msgr2=0x7f0d5c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 0x7f0d5c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 -- 172.21.15.114:0/464179887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50009550 con 0x7f0d5c06c000 2023-11-24T06:27:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5bfff700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 0x7f0d5c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d5b7fe700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000be70 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d500036a0 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c15f790 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50003a30 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.031+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c0ce410 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.032+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c0753f0 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.032+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0d50003b90 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.033+0000 7f0d43fff700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 0x7f0d4806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.033+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d5008b1d0 con 0x7f0d5c06c000 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.033+0000 7f0d5affd700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 0x7f0d4806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.033+0000 7f0d5affd700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 0x7f0d4806d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0d0930 tx=0x7f0d4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.039 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.037+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d50056000 con 0x7f0d5c06c000 2023-11-24T06:27:59.260 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.257+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d5c0686e0 con 0x7f0d4806b080 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.259+0000 7f0d43fff700 1 -- 172.21.15.114:0/464179887 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0d5c0686e0 con 0x7f0d4806b080 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephfs" 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2023-11-24T06:27:59.261 INFO:teuthology.orchestra.run.smithi114.stdout:] 2023-11-24T06:27:59.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 msgr2=0x7f0d4806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.263 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 0x7f0d4806d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0d0930 tx=0x7f0d4c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 msgr2=0x7f0d5c142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d50004750 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 shutdown_connections 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d4806b080 0x7f0d4806d540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d5c15e080 0x7f0d5c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d5c06c000 0x7f0d5c142050 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 --2- 172.21.15.114:0/464179887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d5c0af4d0 0x7f0d5c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 >> 172.21.15.114:0/464179887 conn(0x7f0d5c071b90 msgr2=0x7f0d5c06eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 shutdown_connections 2023-11-24T06:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.261+0000 7f0d6216d700 1 -- 172.21.15.114:0/464179887 wait complete. 2023-11-24T06:27:59.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:27:59.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.396+0000 7ff271aaa700 1 Processor -- start 2023-11-24T06:27:59.428 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.396+0000 7ff271aaa700 1 -- start start 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff271aaa700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff26c0b49a0 0x7ff26c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff271aaa700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff26c06e370 0x7ff26c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff271aaa700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c15e9f0 con 0x7ff26c06e880 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff271aaa700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0b52c0 con 0x7ff26c06e370 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff271aaa700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c15ec90 con 0x7ff26c0b49a0 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff26c06e370 0x7ff26c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26bfff700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff26c06e880 0x7ff26c06ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58098/0 (socket says 172.21.15.114:58098) 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26bfff700 1 -- 172.21.15.114:0/3870459563 learned_addr learned my addr 172.21.15.114:0/3870459563 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:27:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26b7fe700 1 --2- 172.21.15.114:0/3870459563 >> v2:172.21.15.172:3300/0 conn(0x7ff26c0b49a0 0x7ff26c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 -- 172.21.15.114:0/3870459563 >> v2:172.21.15.172:3300/0 conn(0x7ff26c0b49a0 msgr2=0x7ff26c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 --2- 172.21.15.114:0/3870459563 >> v2:172.21.15.172:3300/0 conn(0x7ff26c0b49a0 0x7ff26c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 -- 172.21.15.114:0/3870459563 >> v1:172.21.15.114:6789/0 conn(0x7ff26c06e880 legacy=0x7ff26c06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 -- 172.21.15.114:0/3870459563 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c15ef70 con 0x7ff26c06e370 2023-11-24T06:27:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26a7fc700 1 -- 172.21.15.114:0/3870459563 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283775308 0 0) 0x7ff26c15e9f0 con 0x7ff26c06e880 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26affd700 1 --2- 172.21.15.114:0/3870459563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b4d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff26c072f40 tx=0x7ff260009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af5917b34879e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26a7fc700 1 -- 172.21.15.114:0/3870459563 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260011830 con 0x7ff26c06e370 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26a7fc700 1 -- 172.21.15.114:0/3870459563 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260011990 con 0x7ff26c06e370 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.397+0000 7ff26a7fc700 1 -- 172.21.15.114:0/3870459563 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260011d20 con 0x7ff26c06e370 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 -- 172.21.15.114:0/3870459563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 msgr2=0x7ff26c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.431 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 --2- 172.21.15.114:0/3870459563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b4d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff26c072f40 tx=0x7ff260009b20 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 -- 172.21.15.114:0/3870459563 shutdown_connections 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 --2- 172.21.15.114:0/3870459563 >> v2:172.21.15.172:3300/0 conn(0x7ff26c0b49a0 0x7ff26c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 --2- 172.21.15.114:0/3870459563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b4d80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 -- 172.21.15.114:0/3870459563 >> 172.21.15.114:0/3870459563 conn(0x7ff26c0b28f0 msgr2=0x7ff26c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 -- 172.21.15.114:0/3870459563 shutdown_connections 2023-11-24T06:27:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.398+0000 7ff271aaa700 1 -- 172.21.15.114:0/3870459563 wait complete. 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.399+0000 7ff271aaa700 1 Processor -- start 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.399+0000 7ff271aaa700 1 -- start start 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26c06e880 0x7ff26c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff26c0b49a0 0x7ff26c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.433 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c159e40 con 0x7ff26c06e370 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0af0d0 con 0x7ff26c06e880 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff271aaa700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0af3b0 con 0x7ff26c0b49a0 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42392/0 (socket says 172.21.15.114:42392) 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 -- 172.21.15.114:0/115485981 learned_addr learned my addr 172.21.15.114:0/115485981 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26bfff700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff26c0b49a0 0x7ff26c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26affd700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26c06e880 0x7ff26c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 -- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff26c0b49a0 msgr2=0x7ff26c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff26c0b49a0 0x7ff26c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 -- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26c06e880 msgr2=0x7ff26c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26c06e880 0x7ff26c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.400+0000 7ff26b7fe700 1 -- 172.21.15.114:0/115485981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600094a0 con 0x7ff26c06e370 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff26b7fe700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e810 tx=0x7ff25c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c013610 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c013770 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c0af690 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c013b20 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.401+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c15aaf0 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.402+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c15a400 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.402+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff25c013c80 con 0x7ff26c06e370 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.402+0000 7ff253fff700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 0x7ff25806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.403+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff25c0896e0 con 0x7ff26c06e370 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.403+0000 7ff26affd700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 0x7ff25806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.403+0000 7ff26affd700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 0x7ff25806d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff26c072f40 tx=0x7ff26001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.407+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff25c054490 con 0x7ff26c06e370 2023-11-24T06:27:59.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.597+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff26c068fb0 con 0x7ff26c06e370 2023-11-24T06:27:59.599 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.597+0000 7ff253fff700 1 -- 172.21.15.114:0/115485981 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff25c057ab0 con 0x7ff26c06e370 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 msgr2=0x7ff25806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 0x7ff25806d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff26c072f40 tx=0x7ff26001c040 comp rx=0 tx=0).stop 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 msgr2=0x7ff26c0b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00e810 tx=0x7ff25c00ebd0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 shutdown_connections 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff25806b080 0x7ff25806d540 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff26c0b49a0 0x7ff26c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff26c06e370 0x7ff26c0b00f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 --2- 172.21.15.114:0/115485981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26c06e880 0x7ff26c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 >> 172.21.15.114:0/115485981 conn(0x7ff26c0b28f0 msgr2=0x7ff26c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 shutdown_connections 2023-11-24T06:27:59.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.599+0000 7ff271aaa700 1 -- 172.21.15.114:0/115485981 wait complete. 2023-11-24T06:27:59.614 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:27:59.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:59.656 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:59.703 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:27:59.703 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:59.703 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:59.722 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:27:59.722 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:59.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:59.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.938+0000 7ffa71ac8700 1 Processor -- start 2023-11-24T06:27:59.941 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 -- start start 2023-11-24T06:27:59.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffa6c15e080 0x7ffa6c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c0af4d0 0x7ffa6c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffa6c06c000 0x7ffa6c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.943 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c0af0b0 con 0x7ffa6c06c000 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c06c9f0 con 0x7ffa6c15e080 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa71ac8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0b05a0 con 0x7ffa6c0af4d0 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa6bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffa6c06c000 0x7ffa6c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa6bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffa6c06c000 0x7ffa6c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42404/0 (socket says 172.21.15.114:42404) 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.939+0000 7ffa6bfff700 1 -- 172.21.15.114:0/1750898914 learned_addr learned my addr 172.21.15.114:0/1750898914 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6affd700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c0af4d0 0x7ffa6c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6b7fe700 1 --2- 172.21.15.114:0/1750898914 >> v2:172.21.15.172:3300/0 conn(0x7ffa6c15e080 0x7ffa6c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 -- 172.21.15.114:0/1750898914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c0af4d0 msgr2=0x7ffa6c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c0af4d0 0x7ffa6c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 -- 172.21.15.114:0/1750898914 >> v2:172.21.15.172:3300/0 conn(0x7ffa6c15e080 msgr2=0x7ffa6c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 --2- 172.21.15.114:0/1750898914 >> v2:172.21.15.172:3300/0 conn(0x7ffa6c15e080 0x7ffa6c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 -- 172.21.15.114:0/1750898914 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0ae430 con 0x7ffa6c06c000 2023-11-24T06:27:59.944 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6bfff700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c06c4b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072a90 tx=0x7ffa6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487567f18621f13b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6a7fc700 1 -- 172.21.15.114:0/1750898914 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400e070 con 0x7ffa6c06c000 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6a7fc700 1 -- 172.21.15.114:0/1750898914 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6400b980 con 0x7ffa6c06c000 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.940+0000 7ffa6a7fc700 1 -- 172.21.15.114:0/1750898914 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400bd30 con 0x7ffa6c06c000 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1750898914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 msgr2=0x7ffa6c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c06c4b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c072a90 tx=0x7ffa6400b330 comp rx=0 tx=0).stop 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1750898914 shutdown_connections 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c06c4b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1750898914 >> v2:172.21.15.172:3300/0 conn(0x7ffa6c15e080 0x7ffa6c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1750898914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c0af4d0 0x7ffa6c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1750898914 >> 172.21.15.114:0/1750898914 conn(0x7ffa6c071b90 msgr2=0x7ffa6c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.945 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1750898914 shutdown_connections 2023-11-24T06:27:59.946 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.941+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1750898914 wait complete. 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.942+0000 7ffa71ac8700 1 Processor -- start 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.942+0000 7ffa71ac8700 1 -- start start 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa6c0af4d0 0x7ffa6c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 0x7ffa6c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c160170 con 0x7ffa6c06c000 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c155b70 con 0x7ffa6c0af4d0 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa71ac8700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c155e50 con 0x7ffa6c15e080 2023-11-24T06:27:59.951 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42406/0 (socket says 172.21.15.114:42406) 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 -- 172.21.15.114:0/1524335311 learned_addr learned my addr 172.21.15.114:0/1524335311 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6affd700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa6c0af4d0 0x7ffa6c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6bfff700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 0x7ffa6c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 -- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 msgr2=0x7ffa6c160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 0x7ffa6c160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 -- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa6c0af4d0 msgr2=0x7ffa6c156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa6c0af4d0 0x7ffa6c156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 -- 172.21.15.114:0/1524335311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009b30 con 0x7ffa6c06c000 2023-11-24T06:27:59.952 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6bfff700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 0x7ffa6c160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa6b7fe700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00d800 tx=0x7ffa5c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.943+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0125f0 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.944+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c012750 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.944+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c156130 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.944+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c012b00 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.944+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c1566f0 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.945+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c0753f0 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.945+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ffa5c012c60 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.945+0000 7ffa53fff700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 0x7ffa5806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.945+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa5c089b70 con 0x7ffa6c06c000 2023-11-24T06:27:59.953 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.945+0000 7ffa6affd700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 0x7ffa5806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.946+0000 7ffa6affd700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 0x7ffa5806d540 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c153330 tx=0x7ffa60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.954 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:27:59.950+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa5c054a30 con 0x7ffa6c06c000 2023-11-24T06:28:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.140+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa6c068670 con 0x7ffa6c06c000 2023-11-24T06:28:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.141+0000 7ffa53fff700 1 -- 172.21.15.114:0/1524335311 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7ffa5c058050 con 0x7ffa6c06c000 2023-11-24T06:28:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:00.144 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.143+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 msgr2=0x7ffa5806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.143+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 0x7ffa5806d540 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c153330 tx=0x7ffa60008040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.143+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 msgr2=0x7ffa6c155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.143+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00d800 tx=0x7ffa5c00db10 comp rx=0 tx=0).stop 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 shutdown_connections 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ffa5806b080 0x7ffa5806d540 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa6c15e080 0x7ffa6c160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa6c06c000 0x7ffa6c155540 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 --2- 172.21.15.114:0/1524335311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa6c0af4d0 0x7ffa6c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 >> 172.21.15.114:0/1524335311 conn(0x7ffa6c071b90 msgr2=0x7ffa6c15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 shutdown_connections 2023-11-24T06:28:00.147 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.144+0000 7ffa71ac8700 1 -- 172.21.15.114:0/1524335311 wait complete. 2023-11-24T06:28:00.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:41.963773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:43.243874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:00.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:00.282 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.280+0000 7f0a1d976700 1 Processor -- start 2023-11-24T06:28:00.283 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.280+0000 7f0a1d976700 1 -- start start 2023-11-24T06:28:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a1d976700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a1815e140 0x7f0a180d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a1d976700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 0x7f0a180d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a1d976700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a180b80e0 con 0x7f0a180b98c0 2023-11-24T06:28:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a1d976700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1815e520 con 0x7f0a1815e140 2023-11-24T06:28:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a1d976700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180d2930 con 0x7f0a180d39d0 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a1815e140 0x7f0a180d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a1815e140 0x7f0a180d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42414/0 (socket says 172.21.15.114:42414) 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 -- 172.21.15.114:0/2695246029 learned_addr learned my addr 172.21.15.114:0/2695246029 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a167fc700 1 --2- 172.21.15.114:0/2695246029 >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 0x7f0a180d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 -- 172.21.15.114:0/2695246029 >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 msgr2=0x7f0a180d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 --2- 172.21.15.114:0/2695246029 >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 0x7f0a180d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 -- 172.21.15.114:0/2695246029 >> v1:172.21.15.172:6789/0 conn(0x7f0a180b98c0 legacy=0x7f0a180b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a16ffd700 1 -- 172.21.15.114:0/2695246029 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180d2c10 con 0x7f0a1815e140 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.281+0000 7f0a167fc700 1 --2- 172.21.15.114:0/2695246029 >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 0x7f0a180d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a16ffd700 1 --2- 172.21.15.114:0/2695246029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1815e140 0x7f0a180d3ec0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a180d4b20 tx=0x7f0a0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866cb1d07a6d7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a15ffb700 1 -- 172.21.15.114:0/2695246029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004230 con 0x7f0a1815e140 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a15ffb700 1 -- 172.21.15.114:0/2695246029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c004390 con 0x7f0a1815e140 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a15ffb700 1 -- 172.21.15.114:0/2695246029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0036a0 con 0x7f0a1815e140 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 -- 172.21.15.114:0/2695246029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1815e140 msgr2=0x7f0a180d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 --2- 172.21.15.114:0/2695246029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1815e140 0x7f0a180d3ec0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a180d4b20 tx=0x7f0a0c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 -- 172.21.15.114:0/2695246029 shutdown_connections 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 --2- 172.21.15.114:0/2695246029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1815e140 0x7f0a180d3ec0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 --2- 172.21.15.114:0/2695246029 >> v2:172.21.15.172:3300/0 conn(0x7f0a180d39d0 0x7f0a180d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 -- 172.21.15.114:0/2695246029 >> 172.21.15.114:0/2695246029 conn(0x7f0a1806b410 msgr2=0x7f0a1806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 -- 172.21.15.114:0/2695246029 shutdown_connections 2023-11-24T06:28:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.282+0000 7f0a1d976700 1 -- 172.21.15.114:0/2695246029 wait complete. 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.283+0000 7f0a1d976700 1 Processor -- start 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 -- start start 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a180d39d0 0x7f0a180b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a1815e140 0x7f0a180b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f4da0 con 0x7f0a180b98c0 2023-11-24T06:28:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f3ed0 con 0x7f0a1815e140 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a1d976700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181f4190 con 0x7f0a180d39d0 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.284+0000 7f0a16ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42420/0 (socket says 172.21.15.114:42420) 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 -- 172.21.15.114:0/1725178932 learned_addr learned my addr 172.21.15.114:0/1725178932 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a167fc700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a180d39d0 0x7f0a180b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a177fe700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a1815e140 0x7f0a180b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 -- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a180d39d0 msgr2=0x7f0a180b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a180d39d0 0x7f0a180b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 -- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a1815e140 msgr2=0x7f0a180b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a1815e140 0x7f0a180b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 -- 172.21.15.114:0/1725178932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009550 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a16ffd700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009130 tx=0x7f0a0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003de0 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c011630 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a181f4470 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0119a0 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.285+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181f4a30 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.286+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a181f5080 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.287+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0a0c011b00 con 0x7f0a180b98c0 2023-11-24T06:28:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.287+0000 7f09fffff700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.287+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a0c08ab90 con 0x7f0a180b98c0 2023-11-24T06:28:00.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.287+0000 7f0a167fc700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.287+0000 7f0a167fc700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0a180b2010 tx=0x7f0a08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.293 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.291+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a0c055a50 con 0x7f0a180b98c0 2023-11-24T06:28:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.479+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a180b80e0 con 0x7f0a180b98c0 2023-11-24T06:28:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.480+0000 7f09fffff700 1 -- 172.21.15.114:0/1725178932 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a180b80e0 con 0x7f0a180b98c0 2023-11-24T06:28:00.482 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:28:00.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 msgr2=0x7f0a0406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0a180b2010 tx=0x7f0a08008040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 msgr2=0x7f0a1815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.484 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009130 tx=0x7f0a0c004750 comp rx=0 tx=0).stop 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 shutdown_connections 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0a0406b080 0x7f0a0406d540 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a1815e140 0x7f0a180b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a180b98c0 0x7f0a1815bbb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 --2- 172.21.15.114:0/1725178932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a180d39d0 0x7f0a180b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 >> 172.21.15.114:0/1725178932 conn(0x7f0a1806b410 msgr2=0x7f0a180b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 shutdown_connections 2023-11-24T06:28:00.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.482+0000 7f0a1d976700 1 -- 172.21.15.114:0/1725178932 wait complete. 2023-11-24T06:28:00.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:00.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.617+0000 7f6614d08700 1 Processor -- start 2023-11-24T06:28:00.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.617+0000 7f6614d08700 1 -- start start 2023-11-24T06:28:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f6614d08700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f66101587f0 0x7f661006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f6614d08700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f661015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f6614d08700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661006b530 con 0x7f661015b370 2023-11-24T06:28:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f6614d08700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66100ae5e0 con 0x7f66101587f0 2023-11-24T06:28:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f6614d08700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661015be30 con 0x7f66100b4370 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f660dd9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f661015b370 0x7f66100b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58138/0 (socket says 172.21.15.114:58138) 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f660dd9b700 1 -- 172.21.15.114:0/515911635 learned_addr learned my addr 172.21.15.114:0/515911635 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f660ed9d700 1 --2- 172.21.15.114:0/515911635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f661015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.618+0000 7f660e59c700 1 --2- 172.21.15.114:0/515911635 >> v2:172.21.15.172:3300/0 conn(0x7f66101587f0 0x7f661006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331670221 0 0) 0x7f661006b530 con 0x7f661015b370 2023-11-24T06:28:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f80035b0 con 0x7f661015b370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2812685132 0 0) 0x7f65f80035b0 con 0x7f661015b370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660ed9d700 1 -- 172.21.15.114:0/515911635 >> v2:172.21.15.172:3300/0 conn(0x7f66101587f0 msgr2=0x7f661006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660ed9d700 1 --2- 172.21.15.114:0/515911635 >> v2:172.21.15.172:3300/0 conn(0x7f66101587f0 0x7f661006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660ed9d700 1 -- 172.21.15.114:0/515911635 >> v1:172.21.15.114:6789/0 conn(0x7f661015b370 legacy=0x7f66100b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660ed9d700 1 -- 172.21.15.114:0/515911635 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661015c110 con 0x7f66100b4370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660ed9d700 1 --2- 172.21.15.114:0/515911635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f661015b800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6608009d90 tx=0x7f660800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405ad3ab1492f90a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800e070 con 0x7f66100b4370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608003a20 con 0x7f66100b4370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.619+0000 7f660d59a700 1 -- 172.21.15.114:0/515911635 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600001670 con 0x7f66100b4370 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 -- 172.21.15.114:0/515911635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 msgr2=0x7f661015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 --2- 172.21.15.114:0/515911635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f661015b800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6608009d90 tx=0x7f660800b330 comp rx=0 tx=0).stop 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 -- 172.21.15.114:0/515911635 shutdown_connections 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 --2- 172.21.15.114:0/515911635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f661015b800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 --2- 172.21.15.114:0/515911635 >> v2:172.21.15.172:3300/0 conn(0x7f66101587f0 0x7f661006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 -- 172.21.15.114:0/515911635 >> 172.21.15.114:0/515911635 conn(0x7f66100d0bf0 msgr2=0x7f66100d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 -- 172.21.15.114:0/515911635 shutdown_connections 2023-11-24T06:28:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.620+0000 7f6614d08700 1 -- 172.21.15.114:0/515911635 wait complete. 2023-11-24T06:28:00.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.621+0000 7f6614d08700 1 Processor -- start 2023-11-24T06:28:00.627 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.621+0000 7f6614d08700 1 -- start start 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f6610149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f661015b370 0x7f661014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100d0010 con 0x7f66101587f0 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661014bc40 con 0x7f661015b370 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f6614d08700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661014beb0 con 0x7f66100b4370 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42436/0 (socket says 172.21.15.114:42436) 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 -- 172.21.15.114:0/3076777260 learned_addr learned my addr 172.21.15.114:0/3076777260 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660ed9d700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f661015b370 0x7f661014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660e59c700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f6610149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 -- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 msgr2=0x7f6610149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f6610149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 -- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f661015b370 msgr2=0x7f661014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f661015b370 0x7f661014b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660dd9b700 1 -- 172.21.15.114:0/3076777260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608009a70 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.622+0000 7f660e59c700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f6610149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f660dd9b700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f660000c750 tx=0x7f660000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600004c80 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6600004de0 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661014c190 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66000072f0 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.623+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610149830 con 0x7f66101587f0 2023-11-24T06:28:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.624+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661014c6b0 con 0x7f66101587f0 2023-11-24T06:28:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.624+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6600007450 con 0x7f66101587f0 2023-11-24T06:28:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.625+0000 7f65feffd700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 0x7f65f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.625+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6600089750 con 0x7f66101587f0 2023-11-24T06:28:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.625+0000 7f660e59c700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 0x7f65f006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.625+0000 7f660e59c700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 0x7f65f006d540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6604009b60 tx=0x7f6604008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.629+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6600054580 con 0x7f66101587f0 2023-11-24T06:28:00.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.821+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6610068fb0 con 0x7f66101587f0 2023-11-24T06:28:00.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.822+0000 7f65feffd700 1 -- 172.21.15.114:0/3076777260 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6600007700 con 0x7f66101587f0 2023-11-24T06:28:00.824 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:28:00.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 msgr2=0x7f65f006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 0x7f65f006d540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6604009b60 tx=0x7f6604008040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 msgr2=0x7f66101f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f660000c750 tx=0x7f660000cb10 comp rx=0 tx=0).stop 2023-11-24T06:28:00.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 shutdown_connections 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f65f006b080 0x7f65f006d540 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f661015b370 0x7f661014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66100b4370 0x7f6610149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 --2- 172.21.15.114:0/3076777260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66101587f0 0x7f66101f7c40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 >> 172.21.15.114:0/3076777260 conn(0x7f66100d0bf0 msgr2=0x7f6610156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 shutdown_connections 2023-11-24T06:28:00.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.824+0000 7f6614d08700 1 -- 172.21.15.114:0/3076777260 wait complete. 2023-11-24T06:28:00.838 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:00.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.959+0000 7f13e0719700 1 Processor -- start 2023-11-24T06:28:00.962 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.959+0000 7f13e0719700 1 -- start start 2023-11-24T06:28:00.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.960+0000 7f13e0719700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f13d815e290 0x7f13d80ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.960+0000 7f13e0719700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d806e400 con 0x7f13d80ad870 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.960+0000 7f13e0719700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d806e630 con 0x7f13d80aedc0 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.960+0000 7f13e0719700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d806fde0 con 0x7f13d815e290 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13decb6700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f13d80ad870 0x7f13d806f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58154/0 (socket says 172.21.15.114:58154) 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13decb6700 1 -- 172.21.15.114:0/4136101687 learned_addr learned my addr 172.21.15.114:0/4136101687 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13ddcb4700 1 --2- 172.21.15.114:0/4136101687 >> v2:172.21.15.172:3300/0 conn(0x7f13d815e290 0x7f13d80ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265126198 0 0) 0x7f13d806e400 con 0x7f13d80ad870 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c40035b0 con 0x7f13d80ad870 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783391414 0 0) 0x7f13c40035b0 con 0x7f13d80ad870 2023-11-24T06:28:00.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 >> v2:172.21.15.172:3300/0 conn(0x7f13d815e290 msgr2=0x7f13d80ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 --2- 172.21.15.114:0/4136101687 >> v2:172.21.15.172:3300/0 conn(0x7f13d815e290 0x7f13d80ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 >> v1:172.21.15.172:6789/0 conn(0x7f13d80aedc0 legacy=0x7f13d80af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d806e910 con 0x7f13d80ad870 2023-11-24T06:28:00.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f13d00019f0 con 0x7f13d80ad870 2023-11-24T06:28:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 >> v1:172.21.15.114:6789/0 conn(0x7f13d80ad870 legacy=0x7f13d806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13c4003a00 0x7f13c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4004850 0x7f13c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 0x7f13c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d806fde0 con 0x7f13c4005210 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c400c1b0 con 0x7f13c4004850 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c400c370 con 0x7f13c4003a00 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13decb6700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 0x7f13c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13ddcb4700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4004850 0x7f13c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.961+0000 7f13de4b5700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13c4003a00 0x7f13c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 -- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13c4003a00 msgr2=0x7f13c4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13c4003a00 0x7f13c4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 -- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4004850 msgr2=0x7f13c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4004850 0x7f13c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 -- 172.21.15.114:0/4136101687 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40035b0 con 0x7f13c4005210 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13de4b5700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13c4003a00 0x7f13c4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 -- 172.21.15.114:0/4136101687 reap_dead start 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13decb6700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 0x7f13c400ba10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a830 tx=0x7f13c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4762916a92e8d81b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000a2f0 con 0x7f13c4005210 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d0012600 con 0x7f13c4005210 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13dd4b3700 1 -- 172.21.15.114:0/4136101687 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d00129b0 con 0x7f13c4005210 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13e0719700 1 -- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 msgr2=0x7f13c400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13e0719700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 0x7f13c400ba10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a830 tx=0x7f13c4005880 comp rx=0 tx=0).stop 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13e0719700 1 -- 172.21.15.114:0/4136101687 shutdown_connections 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.962+0000 7f13e0719700 1 --2- 172.21.15.114:0/4136101687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13c4005210 0x7f13c400ba10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.963+0000 7f13e0719700 1 -- 172.21.15.114:0/4136101687 >> 172.21.15.114:0/4136101687 conn(0x7f13d80b2050 msgr2=0x7f13d80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.963+0000 7f13e0719700 1 -- 172.21.15.114:0/4136101687 shutdown_connections 2023-11-24T06:28:00.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.963+0000 7f13e0719700 1 -- 172.21.15.114:0/4136101687 wait complete. 2023-11-24T06:28:00.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.964+0000 7f13e0719700 1 Processor -- start 2023-11-24T06:28:00.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.964+0000 7f13e0719700 1 -- start start 2023-11-24T06:28:00.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4005210 0x7f13d8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d81419d0 0x7f13d8141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81423c0 con 0x7f13d80c58f0 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d813fa10 con 0x7f13c4005210 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13e0719700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80cf0a0 con 0x7f13d81419d0 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42454/0 (socket says 172.21.15.114:42454) 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 -- 172.21.15.114:0/1080131967 learned_addr learned my addr 172.21.15.114:0/1080131967 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13de4b5700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4005210 0x7f13d8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.975 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13decb6700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d81419d0 0x7f13d8141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 -- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d81419d0 msgr2=0x7f13d8141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d81419d0 0x7f13d8141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 -- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4005210 msgr2=0x7f13d8152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4005210 0x7f13d8152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.965+0000 7f13ddcb4700 1 -- 172.21.15.114:0/1080131967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80cf380 con 0x7f13d80c58f0 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13ddcb4700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f13d40119a0 tx=0x7f13d4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d400f940 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d400faa0 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d813fcb0 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d400fe30 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.966+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81528a0 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.967+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8068d90 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.967+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f13d4014440 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.967+0000 7f13ceffd700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 0x7f13bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.968+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13d408a050 con 0x7f13d80c58f0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.968+0000 7f13de4b5700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 0x7f13bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.968+0000 7f13de4b5700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 0x7f13bc06d540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005530 tx=0x7f13c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.977 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:00.972+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13d4054e80 con 0x7f13d80c58f0 2023-11-24T06:28:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.160+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13d8068fb0 con 0x7f13d80c58f0 2023-11-24T06:28:01.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.160+0000 7f13ceffd700 1 -- 172.21.15.114:0/1080131967 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13d40584a0 con 0x7f13d80c58f0 2023-11-24T06:28:01.163 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:28:01.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 msgr2=0x7f13bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 0x7f13bc06d540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005530 tx=0x7f13c8009040 comp rx=0 tx=0).stop 2023-11-24T06:28:01.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 msgr2=0x7f13d80ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f13d40119a0 tx=0x7f13d4011d60 comp rx=0 tx=0).stop 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 shutdown_connections 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f13bc06b080 0x7f13bc06d540 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d81419d0 0x7f13d8141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13c4005210 0x7f13d8152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 --2- 172.21.15.114:0/1080131967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f13d80c58f0 0x7f13d80ce790 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 >> 172.21.15.114:0/1080131967 conn(0x7f13d80b2050 msgr2=0x7f13d8160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 shutdown_connections 2023-11-24T06:28:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.162+0000 7f13e0719700 1 -- 172.21.15.114:0/1080131967 wait complete. 2023-11-24T06:28:01.177 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.298+0000 7f97a217c700 1 Processor -- start 2023-11-24T06:28:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.298+0000 7f97a217c700 1 -- start start 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f979c0af4d0 0x7f979c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f979c06c000 0x7f979c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0af0b0 con 0x7f979c06c000 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c06c9f0 con 0x7f979c0af4d0 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f97a217c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0b05a0 con 0x7f979c15e080 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f979c06c000 0x7f979c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f979c06c000 0x7f979c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42466/0 (socket says 172.21.15.114:42466) 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 -- 172.21.15.114:0/967198222 learned_addr learned my addr 172.21.15.114:0/967198222 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979affd700 1 --2- 172.21.15.114:0/967198222 >> v2:172.21.15.172:3300/0 conn(0x7f979c0af4d0 0x7f979c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979b7fe700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 -- 172.21.15.114:0/967198222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 msgr2=0x7f979c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 -- 172.21.15.114:0/967198222 >> v2:172.21.15.172:3300/0 conn(0x7f979c0af4d0 msgr2=0x7f979c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 --2- 172.21.15.114:0/967198222 >> v2:172.21.15.172:3300/0 conn(0x7f979c0af4d0 0x7f979c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 -- 172.21.15.114:0/967198222 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0ae430 con 0x7f979c06c000 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979bfff700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c06c000 0x7f979c06c4b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f979c06d380 tx=0x7f979400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc276fb931be61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.299+0000 7f979a7fc700 1 -- 172.21.15.114:0/967198222 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400e070 con 0x7f979c06c000 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f979a7fc700 1 -- 172.21.15.114:0/967198222 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979400b980 con 0x7f979c06c000 2023-11-24T06:28:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f979a7fc700 1 -- 172.21.15.114:0/967198222 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979400bd30 con 0x7f979c06c000 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 -- 172.21.15.114:0/967198222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c06c000 msgr2=0x7f979c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c06c000 0x7f979c06c4b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f979c06d380 tx=0x7f979400b330 comp rx=0 tx=0).stop 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 -- 172.21.15.114:0/967198222 shutdown_connections 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 --2- 172.21.15.114:0/967198222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c06c000 0x7f979c06c4b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 --2- 172.21.15.114:0/967198222 >> v2:172.21.15.172:3300/0 conn(0x7f979c0af4d0 0x7f979c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 -- 172.21.15.114:0/967198222 >> 172.21.15.114:0/967198222 conn(0x7f979c071b90 msgr2=0x7f979c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 -- 172.21.15.114:0/967198222 shutdown_connections 2023-11-24T06:28:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.300+0000 7f97a217c700 1 -- 172.21.15.114:0/967198222 wait complete. 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.301+0000 7f97a217c700 1 Processor -- start 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.301+0000 7f97a217c700 1 -- start start 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f979c06c000 0x7f979c157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c158950 con 0x7f979c0af4d0 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c157cd0 con 0x7f979c06c000 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f97a217c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c157fb0 con 0x7f979c15e080 2023-11-24T06:28:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f979affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f979affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42470/0 (socket says 172.21.15.114:42470) 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.302+0000 7f979affd700 1 -- 172.21.15.114:0/1298626427 learned_addr learned my addr 172.21.15.114:0/1298626427 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979b7fe700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f979c06c000 0x7f979c157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979bfff700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 -- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 msgr2=0x7f979c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 -- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f979c06c000 msgr2=0x7f979c157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f979c06c000 0x7f979c157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 -- 172.21.15.114:0/1298626427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009b30 con 0x7f979c0af4d0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f979affd700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c7f0 tx=0x7f978c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c004220 con 0x7f979c0af4d0 2023-11-24T06:28:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c004380 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c158290 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0036a0 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.303+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c156a00 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.304+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c0753f0 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.304+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f978c003800 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.305+0000 7f9783fff700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 0x7f978806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.305+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f978c089580 con 0x7f979c0af4d0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.305+0000 7f979b7fe700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 0x7f978806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.305+0000 7f979b7fe700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 0x7f978806d540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b5a10 tx=0x7f9790008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.311 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.309+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f978c0542b0 con 0x7f979c0af4d0 2023-11-24T06:28:01.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.515+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f979c067f60 con 0x7f979c0af4d0 2023-11-24T06:28:01.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.515+0000 7f9783fff700 1 -- 172.21.15.114:0/1298626427 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f978c0578d0 con 0x7f979c0af4d0 2023-11-24T06:28:01.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 msgr2=0x7f978806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 0x7f978806d540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f979c0b5a10 tx=0x7f9790008040 comp rx=0 tx=0).stop 2023-11-24T06:28:01.521 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 msgr2=0x7f979c15ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c7f0 tx=0x7f978c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 shutdown_connections 2023-11-24T06:28:01.522 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f978806b080 0x7f978806d540 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f979c15e080 0x7f979c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f979c06c000 0x7f979c157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 --2- 172.21.15.114:0/1298626427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f979c0af4d0 0x7f979c15ffa0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 >> 172.21.15.114:0/1298626427 conn(0x7f979c071b90 msgr2=0x7f979c0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 shutdown_connections 2023-11-24T06:28:01.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.517+0000 7f97a217c700 1 -- 172.21.15.114:0/1298626427 wait complete. 2023-11-24T06:28:01.524 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:28:01.532 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:28:01.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.532 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:01.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:01.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:01.579 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:28:01.596 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:28:01.596 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:01.617 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:28:01.618 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:01.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:01.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.798+0000 7ff947053700 1 Processor -- start 2023-11-24T06:28:01.801 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.798+0000 7ff947053700 1 -- start start 2023-11-24T06:28:01.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff947053700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff94015b200 0x7ff9400b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff947053700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff940164b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff947053700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9400cfd40 con 0x7ff94015ae20 2023-11-24T06:28:01.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff947053700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9400d0020 con 0x7ff94015b200 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff947053700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400affa0 con 0x7ff9400b1a70 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93ffff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff94015b200 0x7ff9400b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff944def700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff94015ae20 0x7ff9400b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58184/0 (socket says 172.21.15.114:58184) 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff944def700 1 -- 172.21.15.114:0/3701269646 learned_addr learned my addr 172.21.15.114:0/3701269646 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff9455f0700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff940164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93f7fe700 1 -- 172.21.15.114:0/3701269646 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590929464 0 0) 0x7ff9400cfd40 con 0x7ff94015ae20 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93f7fe700 1 -- 172.21.15.114:0/3701269646 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9300057c0 con 0x7ff94015ae20 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93ffff700 1 -- 172.21.15.114:0/3701269646 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 msgr2=0x7ff940164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93ffff700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff940164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93ffff700 1 -- 172.21.15.114:0/3701269646 >> v1:172.21.15.114:6789/0 conn(0x7ff94015ae20 legacy=0x7ff9400b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.799+0000 7ff93ffff700 1 -- 172.21.15.114:0/3701269646 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9400b0280 con 0x7ff94015b200 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff9455f0700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff940164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff93ffff700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015b200 0x7ff9400b29a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009840 tx=0x7ff92c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d623da023f00639 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff93f7fe700 1 -- 172.21.15.114:0/3701269646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c011830 con 0x7ff94015b200 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff93f7fe700 1 -- 172.21.15.114:0/3701269646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c011990 con 0x7ff94015b200 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff93f7fe700 1 -- 172.21.15.114:0/3701269646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c011d20 con 0x7ff94015b200 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 -- 172.21.15.114:0/3701269646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015b200 msgr2=0x7ff9400b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015b200 0x7ff9400b29a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009840 tx=0x7ff92c009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 -- 172.21.15.114:0/3701269646 shutdown_connections 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff940164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 --2- 172.21.15.114:0/3701269646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015b200 0x7ff9400b29a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 -- 172.21.15.114:0/3701269646 >> 172.21.15.114:0/3701269646 conn(0x7ff9400b57b0 msgr2=0x7ff9400b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 -- 172.21.15.114:0/3701269646 shutdown_connections 2023-11-24T06:28:01.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.800+0000 7ff947053700 1 -- 172.21.15.114:0/3701269646 wait complete. 2023-11-24T06:28:01.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 Processor -- start 2023-11-24T06:28:01.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 -- start start 2023-11-24T06:28:01.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff9401eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.808 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94015b200 0x7ff9400ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ade80 con 0x7ff94015ae20 2023-11-24T06:28:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.802+0000 7ff947053700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ad270 con 0x7ff94015b200 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff947053700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9400ad550 con 0x7ff9400b1a70 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42484/0 (socket says 172.21.15.114:42484) 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 -- 172.21.15.114:0/755235638 learned_addr learned my addr 172.21.15.114:0/755235638 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff944def700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff9401eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff9455f0700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94015b200 0x7ff9400ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 -- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 msgr2=0x7ff9401eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff9401eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 -- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94015b200 msgr2=0x7ff9400ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94015b200 0x7ff9400ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 -- 172.21.15.114:0/755235638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0094a0 con 0x7ff94015ae20 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93ffff700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0090b0 tx=0x7ff92c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0036a0 con 0x7ff94015ae20 2023-11-24T06:28:01.811 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c01e070 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9400ad830 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003a60 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.803+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401ef500 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.804+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940067f60 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.804+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff92c003bc0 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.805+0000 7ff93d7fa700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 0x7ff92406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.805+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff92c08a9e0 con 0x7ff94015ae20 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.805+0000 7ff944def700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 0x7ff92406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.805+0000 7ff944def700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 0x7ff92406d540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9340011e0 tx=0x7ff934009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.813 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:01.809+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff92c0558a0 con 0x7ff94015ae20 2023-11-24T06:28:02.005 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.002+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9400ad690 con 0x7ff94015ae20 2023-11-24T06:28:02.006 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.003+0000 7ff93d7fa700 1 -- 172.21.15.114:0/755235638 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7ff9400ad690 con 0x7ff94015ae20 2023-11-24T06:28:02.007 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:02.008 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:02.009 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 msgr2=0x7ff92406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 0x7ff92406d540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9340011e0 tx=0x7ff934009040 comp rx=0 tx=0).stop 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 msgr2=0x7ff9400cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0090b0 tx=0x7ff92c004750 comp rx=0 tx=0).stop 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 shutdown_connections 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7ff92406b080 0x7ff92406d540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94015b200 0x7ff9400ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9400b1a70 0x7ff9401eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 --2- 172.21.15.114:0/755235638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff94015ae20 0x7ff9400cb4d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 >> 172.21.15.114:0/755235638 conn(0x7ff9400b57b0 msgr2=0x7ff94006d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 shutdown_connections 2023-11-24T06:28:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.005+0000 7ff947053700 1 -- 172.21.15.114:0/755235638 wait complete. 2023-11-24T06:28:02.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:41.963773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:43.243874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:02.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:02.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.143+0000 7f08a531d700 1 Processor -- start 2023-11-24T06:28:02.146 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.144+0000 7f08a531d700 1 -- start start 2023-11-24T06:28:02.149 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.144+0000 7f08a531d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08a00cf240 0x7f08a00d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.144+0000 7f08a531d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00d1aa0 con 0x7f08a015e1d0 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.144+0000 7f08a531d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00d1cf0 con 0x7f08a00d3970 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.144+0000 7f08a531d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a00d1fd0 con 0x7f08a00cf240 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089effd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f08a015e1d0 0x7f08a00d2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58198/0 (socket says 172.21.15.114:58198) 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089effd700 1 -- 172.21.15.114:0/2648173785 learned_addr learned my addr 172.21.15.114:0/2648173785 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 --2- 172.21.15.114:0/2648173785 >> v2:172.21.15.172:3300/0 conn(0x7f08a00cf240 0x7f08a00d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381713291 0 0) 0x7f08a00d1aa0 con 0x7f08a015e1d0 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08880035b0 con 0x7f08a015e1d0 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1511734706 0 0) 0x7f08880035b0 con 0x7f08a015e1d0 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 >> v2:172.21.15.172:3300/0 conn(0x7f08a00cf240 msgr2=0x7f08a00d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 --2- 172.21.15.114:0/2648173785 >> v2:172.21.15.172:3300/0 conn(0x7f08a00cf240 0x7f08a00d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 >> v1:172.21.15.172:6789/0 conn(0x7f08a00d3970 legacy=0x7f08a00b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a00d22b0 con 0x7f08a015e1d0 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f0894001670 con 0x7f08a015e1d0 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 >> v1:172.21.15.114:6789/0 conn(0x7f08a015e1d0 legacy=0x7f08a00d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888003a00 0x7f0888003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f0888004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0888005210 0x7f088800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00d1fd0 con 0x7f0888004850 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088800c1b0 con 0x7f0888003a00 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088800c370 con 0x7f0888005210 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f0888004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089effd700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888003a00 0x7f0888003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089f7fe700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0888005210 0x7f088800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 -- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0888005210 msgr2=0x7f088800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0888005210 0x7f088800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 -- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888003a00 msgr2=0x7f0888003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888003a00 0x7f0888003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.145+0000 7f089e7fc700 1 -- 172.21.15.114:0/2648173785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880035b0 con 0x7f0888004850 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f089f7fe700 1 -- 172.21.15.114:0/2648173785 reap_dead start 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f089e7fc700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f0888004cd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0890009b30 tx=0x7f0888005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a916ee339c87f005 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089000e6c0 con 0x7f0888004850 2023-11-24T06:28:02.152 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089000e820 con 0x7f0888004850 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f089dffb700 1 -- 172.21.15.114:0/2648173785 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089000eb90 con 0x7f0888004850 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 -- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 msgr2=0x7f0888004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f0888004cd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0890009b30 tx=0x7f0888005880 comp rx=0 tx=0).stop 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 -- 172.21.15.114:0/2648173785 shutdown_connections 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 --2- 172.21.15.114:0/2648173785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f0888004cd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 -- 172.21.15.114:0/2648173785 >> 172.21.15.114:0/2648173785 conn(0x7f08a00d0950 msgr2=0x7f08a00d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 -- 172.21.15.114:0/2648173785 shutdown_connections 2023-11-24T06:28:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.146+0000 7f08a531d700 1 -- 172.21.15.114:0/2648173785 wait complete. 2023-11-24T06:28:02.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 Processor -- start 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 -- start start 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 0x7f08a00c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a01f75b0 0x7f08a01f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00c8a50 con 0x7f0888004850 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a01f80b0 con 0x7f08a01f75b0 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f08a531d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a01f8390 con 0x7f08a00c94a0 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f089effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.148+0000 7f089effd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42502/0 (socket says 172.21.15.114:42502) 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 -- 172.21.15.114:0/1046495478 learned_addr learned my addr 172.21.15.114:0/1046495478 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:02.160 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089e7fc700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 0x7f08a00c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089f7fe700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a01f75b0 0x7f08a01f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 -- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 msgr2=0x7f08a00c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 0x7f08a00c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 -- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a01f75b0 msgr2=0x7f08a01f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a01f75b0 0x7f08a01f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 -- 172.21.15.114:0/1046495478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a00c8ca0 con 0x7f0888004850 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089e7fc700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 0x7f08a00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f089effd700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0894007290 tx=0x7f0894007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894001490 con 0x7f0888004850 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089400f7f0 con 0x7f0888004850 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a00c8f80 con 0x7f0888004850 2023-11-24T06:28:02.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089400fba0 con 0x7f0888004850 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.149+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a014cdf0 con 0x7f0888004850 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.150+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a00c92f0 con 0x7f0888004850 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.150+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f089400fd00 con 0x7f0888004850 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.151+0000 7f08877fe700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 0x7f088c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.151+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0894089940 con 0x7f0888004850 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.151+0000 7f089e7fc700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 0x7f088c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.151+0000 7f089e7fc700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 0x7f088c06d540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f08900090b0 tx=0x7f0890008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.155+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0894054820 con 0x7f0888004850 2023-11-24T06:28:02.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.343+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08a0068fb0 con 0x7f0888004850 2023-11-24T06:28:02.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.344+0000 7f08877fe700 1 -- 172.21.15.114:0/1046495478 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0894057e40 con 0x7f0888004850 2023-11-24T06:28:02.346 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:28:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 msgr2=0x7f088c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.348 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 0x7f088c06d540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f08900090b0 tx=0x7f0890008030 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 msgr2=0x7f08a014c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0894007290 tx=0x7f0894007650 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 shutdown_connections 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f088c06b080 0x7f088c06d540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a01f75b0 0x7f08a01f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0888004850 0x7f08a014c8b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 --2- 172.21.15.114:0/1046495478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a00c94a0 0x7f08a00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 >> 172.21.15.114:0/1046495478 conn(0x7f08a00d0950 msgr2=0x7f08a00b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 shutdown_connections 2023-11-24T06:28:02.349 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.346+0000 7f08a531d700 1 -- 172.21.15.114:0/1046495478 wait complete. 2023-11-24T06:28:02.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:02.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.482+0000 7f968a65b700 1 Processor -- start 2023-11-24T06:28:02.485 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 -- start start 2023-11-24T06:28:02.487 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96840b3250 0x7f968406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840b6fb0 con 0x7f968415e320 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840d4da0 con 0x7f96840b3250 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f968a65b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840d2c10 con 0x7f96840b93c0 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f9688bf8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96840b3250 0x7f968406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f9688bf8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96840b3250 0x7f968406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42514/0 (socket says 172.21.15.114:42514) 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.483+0000 7f9688bf8700 1 -- 172.21.15.114:0/3544090704 learned_addr learned my addr 172.21.15.114:0/3544090704 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f96837fe700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9688bf8700 1 -- 172.21.15.114:0/3544090704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 msgr2=0x7f968406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9688bf8700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9688bf8700 1 -- 172.21.15.114:0/3544090704 >> v1:172.21.15.172:6789/0 conn(0x7f968415e320 legacy=0x7f96840d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9688bf8700 1 -- 172.21.15.114:0/3544090704 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840d2ef0 con 0x7f96840b3250 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9688bf8700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f968406b530 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f967c009e50 tx=0x7f967c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26219e963a1103a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9682ffd700 1 -- 172.21.15.114:0/3544090704 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00e070 con 0x7f96840b3250 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9682ffd700 1 -- 172.21.15.114:0/3544090704 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00b9c0 con 0x7f96840b3250 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.484+0000 7f9682ffd700 1 -- 172.21.15.114:0/3544090704 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00bd70 con 0x7f96840b3250 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 -- 172.21.15.114:0/3544090704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 msgr2=0x7f968406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f968406b530 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f967c009e50 tx=0x7f967c00b330 comp rx=0 tx=0).stop 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 -- 172.21.15.114:0/3544090704 shutdown_connections 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f968406b530 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 --2- 172.21.15.114:0/3544090704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 -- 172.21.15.114:0/3544090704 >> 172.21.15.114:0/3544090704 conn(0x7f96840d3e50 msgr2=0x7f96840d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 -- 172.21.15.114:0/3544090704 shutdown_connections 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.485+0000 7f968a65b700 1 -- 172.21.15.114:0/3544090704 wait complete. 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.486+0000 7f968a65b700 1 Processor -- start 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.486+0000 7f968a65b700 1 -- start start 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 0x7f96840ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c8cc0 con 0x7f96840b3250 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96841596a0 con 0x7f968415e320 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f968a65b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684159960 con 0x7f96840b93c0 2023-11-24T06:28:02.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.493 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42516/0 (socket says 172.21.15.114:42516) 2023-11-24T06:28:02.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 -- 172.21.15.114:0/3538602823 learned_addr learned my addr 172.21.15.114:0/3538602823 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:02.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f96837fe700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.494 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9688bf8700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 0x7f96840ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 -- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 msgr2=0x7f968415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968415aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 -- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 msgr2=0x7f96840ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 0x7f96840ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 -- 172.21.15.114:0/3538602823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c009b30 con 0x7f96840b3250 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f96837fe700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9688bf8700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 0x7f96840ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.487+0000 7f9683fff700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f967800c450 tx=0x7f967800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.488+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780052d0 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.488+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678005430 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.488+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9684159c40 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.488+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780038c0 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.488+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968415a200 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.489+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96840cfc10 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.489+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9678003a20 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.489+0000 7f9680ff9700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 0x7f967006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.489+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9678088c00 con 0x7f96840b3250 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.489+0000 7f96837fe700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 0x7f967006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.490+0000 7f96837fe700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 0x7f967006d570 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96840cf310 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.494+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f967808f080 con 0x7f96840b3250 2023-11-24T06:28:02.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.688+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9684068fb0 con 0x7f96840b3250 2023-11-24T06:28:02.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.688+0000 7f9680ff9700 1 -- 172.21.15.114:0/3538602823 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9678053a30 con 0x7f96840b3250 2023-11-24T06:28:02.691 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:28:02.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 msgr2=0x7f967006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 0x7f967006d570 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96840cf310 tx=0x7f9674008040 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 msgr2=0x7f96841509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f967800c450 tx=0x7f967800c810 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 shutdown_connections 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f967006b0b0 0x7f967006d570 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f968415e320 0x7f96840ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96840b3250 0x7f96841509d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 --2- 172.21.15.114:0/3538602823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96840b93c0 0x7f968415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 >> 172.21.15.114:0/3538602823 conn(0x7f96840d3e50 msgr2=0x7f96840b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.690+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 shutdown_connections 2023-11-24T06:28:02.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.691+0000 7f968a65b700 1 -- 172.21.15.114:0/3538602823 wait complete. 2023-11-24T06:28:02.707 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:02.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.827+0000 7f8daa0bc700 1 Processor -- start 2023-11-24T06:28:02.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.827+0000 7f8daa0bc700 1 -- start start 2023-11-24T06:28:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8daa0bc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 0x7f8da40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8daa0bc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8da40cef40 0x7f8da40cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8daa0bc700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da40d0140 con 0x7f8da415bf50 2023-11-24T06:28:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8daa0bc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da415c330 con 0x7f8da40cef40 2023-11-24T06:28:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8daa0bc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da40cf3d0 con 0x7f8da40c6c30 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8da40cef40 0x7f8da40cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8da40cef40 0x7f8da40cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42528/0 (socket says 172.21.15.114:42528) 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 -- 172.21.15.114:0/2353632759 learned_addr learned my addr 172.21.15.114:0/2353632759 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da2ffd700 1 --2- 172.21.15.114:0/2353632759 >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 0x7f8da40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 -- 172.21.15.114:0/2353632759 >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 msgr2=0x7f8da40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 --2- 172.21.15.114:0/2353632759 >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 0x7f8da40cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 -- 172.21.15.114:0/2353632759 >> v1:172.21.15.172:6789/0 conn(0x7f8da415bf50 legacy=0x7f8da40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 -- 172.21.15.114:0/2353632759 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40d03e0 con 0x7f8da40cef40 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da2ffd700 1 --2- 172.21.15.114:0/2353632759 >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 0x7f8da40cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.828+0000 7f8da3fff700 1 --2- 172.21.15.114:0/2353632759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da40cef40 0x7f8da40cc6c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8da4072420 tx=0x7f8d9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca35b8cff145c08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8da27fc700 1 -- 172.21.15.114:0/2353632759 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00e070 con 0x7f8da40cef40 2023-11-24T06:28:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8da27fc700 1 -- 172.21.15.114:0/2353632759 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c003a20 con 0x7f8da40cef40 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8da27fc700 1 -- 172.21.15.114:0/2353632759 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c003dd0 con 0x7f8da40cef40 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 -- 172.21.15.114:0/2353632759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da40cef40 msgr2=0x7f8da40cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/2353632759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da40cef40 0x7f8da40cc6c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8da4072420 tx=0x7f8d9c00b330 comp rx=0 tx=0).stop 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 -- 172.21.15.114:0/2353632759 shutdown_connections 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/2353632759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da40cef40 0x7f8da40cc6c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/2353632759 >> v2:172.21.15.172:3300/0 conn(0x7f8da40c6c30 0x7f8da40cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 -- 172.21.15.114:0/2353632759 >> 172.21.15.114:0/2353632759 conn(0x7f8da40b3780 msgr2=0x7f8da40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 -- 172.21.15.114:0/2353632759 shutdown_connections 2023-11-24T06:28:02.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.829+0000 7f8daa0bc700 1 -- 172.21.15.114:0/2353632759 wait complete. 2023-11-24T06:28:02.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.830+0000 7f8daa0bc700 1 Processor -- start 2023-11-24T06:28:02.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 -- start start 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 0x7f8da4146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8da40cef40 0x7f8da40b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41f1a90 con 0x7f8da415bf50 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4147a50 con 0x7f8da40c6c30 2023-11-24T06:28:02.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8daa0bc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da4147d30 con 0x7f8da40cef40 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42534/0 (socket says 172.21.15.114:42534) 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.831+0000 7f8da3fff700 1 -- 172.21.15.114:0/1319640471 learned_addr learned my addr 172.21.15.114:0/1319640471 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da37fe700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 0x7f8da4146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da2ffd700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8da40cef40 0x7f8da40b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 -- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8da40cef40 msgr2=0x7f8da40b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8da40cef40 0x7f8da40b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 -- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 msgr2=0x7f8da4146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 0x7f8da4146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 -- 172.21.15.114:0/1319640471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c009b30 con 0x7f8da415bf50 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da37fe700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 0x7f8da4146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8da3fff700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c015040 tx=0x7f8d9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0092e0 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c004240 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da4148010 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0045f0 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.832+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da41485d0 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.833+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da4148760 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.833+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8d9c019070 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.834+0000 7f8d8bfff700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 0x7f8d9006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.834+0000 7f8da37fe700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 0x7f8d9006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.834+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d9c08a920 con 0x7f8da415bf50 2023-11-24T06:28:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.834+0000 7f8da37fe700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 0x7f8d9006d540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009930 tx=0x7f8d98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:02.838+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d9c0556d0 con 0x7f8da415bf50 2023-11-24T06:28:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.027+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8da4068fb0 con 0x7f8da415bf50 2023-11-24T06:28:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.028+0000 7f8d8bfff700 1 -- 172.21.15.114:0/1319640471 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d9c02a020 con 0x7f8da415bf50 2023-11-24T06:28:03.030 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:28:03.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 msgr2=0x7f8d9006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 0x7f8d9006d540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009930 tx=0x7f8d98008040 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 msgr2=0x7f8da40aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c015040 tx=0x7f8d9c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 shutdown_connections 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8d9006b080 0x7f8d9006d540 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8da415bf50 0x7f8da40aea60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8da40c6c30 0x7f8da4146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 --2- 172.21.15.114:0/1319640471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8da40cef40 0x7f8da40b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 >> 172.21.15.114:0/1319640471 conn(0x7f8da40b3780 msgr2=0x7f8da40ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 shutdown_connections 2023-11-24T06:28:03.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.030+0000 7f8daa0bc700 1 -- 172.21.15.114:0/1319640471 wait complete. 2023-11-24T06:28:03.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:03.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.164+0000 7f4dce070700 1 Processor -- start 2023-11-24T06:28:03.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.164+0000 7f4dce070700 1 -- start start 2023-11-24T06:28:03.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dce070700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4dc80b3370 0x7f4dc81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dce070700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4dc80d1c70 0x7f4dc80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dce070700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc80b22d0 con 0x7f4dc815bdd0 2023-11-24T06:28:03.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dce070700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc80b2f50 con 0x7f4dc80b3370 2023-11-24T06:28:03.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dce070700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc80d2640 con 0x7f4dc80d1c70 2023-11-24T06:28:03.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4dc80b3370 0x7f4dc81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4dc80b3370 0x7f4dc81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42540/0 (socket says 172.21.15.114:42540) 2023-11-24T06:28:03.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/2231511998 learned_addr learned my addr 172.21.15.114:0/2231511998 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:03.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/2231511998 >> v2:172.21.15.172:3300/0 conn(0x7f4dc80d1c70 msgr2=0x7f4dc80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 --2- 172.21.15.114:0/2231511998 >> v2:172.21.15.172:3300/0 conn(0x7f4dc80d1c70 0x7f4dc80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/2231511998 >> v1:172.21.15.172:6789/0 conn(0x7f4dc815bdd0 legacy=0x7f4dc80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/2231511998 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80b25b0 con 0x7f4dc80b3370 2023-11-24T06:28:03.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.165+0000 7f4dc6ffd700 1 --2- 172.21.15.114:0/2231511998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80b3370 0x7f4dc81604e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8160a50 tx=0x7f4db8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65af8392b511b065 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dc67fc700 1 -- 172.21.15.114:0/2231511998 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004230 con 0x7f4dc80b3370 2023-11-24T06:28:03.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dc67fc700 1 -- 172.21.15.114:0/2231511998 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8004390 con 0x7f4dc80b3370 2023-11-24T06:28:03.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dc67fc700 1 -- 172.21.15.114:0/2231511998 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db80036a0 con 0x7f4dc80b3370 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 -- 172.21.15.114:0/2231511998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80b3370 msgr2=0x7f4dc81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 --2- 172.21.15.114:0/2231511998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80b3370 0x7f4dc81604e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8160a50 tx=0x7f4db8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 -- 172.21.15.114:0/2231511998 shutdown_connections 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 --2- 172.21.15.114:0/2231511998 >> v2:172.21.15.172:3300/0 conn(0x7f4dc80d1c70 0x7f4dc80d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 --2- 172.21.15.114:0/2231511998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80b3370 0x7f4dc81604e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 -- 172.21.15.114:0/2231511998 >> 172.21.15.114:0/2231511998 conn(0x7f4dc80731a0 msgr2=0x7f4dc809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 -- 172.21.15.114:0/2231511998 shutdown_connections 2023-11-24T06:28:03.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.166+0000 7f4dce070700 1 -- 172.21.15.114:0/2231511998 wait complete. 2023-11-24T06:28:03.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.167+0000 7f4dce070700 1 Processor -- start 2023-11-24T06:28:03.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 -- start start 2023-11-24T06:28:03.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 0x7f4dc80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dc815bdd0 0x7f4dc80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8157810 con 0x7f4dc80d1c70 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80cd870 con 0x7f4dc80b3370 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dce070700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80ccc60 con 0x7f4dc815bdd0 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42546/0 (socket says 172.21.15.114:42546) 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.168+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/1497312387 learned_addr learned my addr 172.21.15.114:0/1497312387 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc77fe700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 0x7f4dc80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc7fff700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dc815bdd0 0x7f4dc80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dc815bdd0 msgr2=0x7f4dc80ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dc815bdd0 0x7f4dc80ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 msgr2=0x7f4dc80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 0x7f4dc80cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 -- 172.21.15.114:0/1497312387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009550 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc77fe700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 0x7f4dc80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dc6ffd700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80716d0 tx=0x7f4db8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8003d80 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db801f070 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc80ccf40 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8011870 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.169+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc80cd500 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.170+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc80cd6c0 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.170+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4db8021030 con 0x7f4dc80d1c70 2023-11-24T06:28:03.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.171+0000 7f4daffff700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 0x7f4db406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.171+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db808b060 con 0x7f4dc80d1c70 2023-11-24T06:28:03.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.171+0000 7f4dc77fe700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 0x7f4db406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.171+0000 7f4dc77fe700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 0x7f4db406d540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009910 tx=0x7f4dbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.175+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4db8056040 con 0x7f4dc80d1c70 2023-11-24T06:28:03.385 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.383+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4dc8067f60 con 0x7f4dc80d1c70 2023-11-24T06:28:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.383+0000 7f4daffff700 1 -- 172.21.15.114:0/1497312387 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4db8059660 con 0x7f4dc80d1c70 2023-11-24T06:28:03.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 msgr2=0x7f4db406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 0x7f4db406d540 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009910 tx=0x7f4dbc008040 comp rx=0 tx=0).stop 2023-11-24T06:28:03.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 msgr2=0x7f4dc81572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80716d0 tx=0x7f4db8004750 comp rx=0 tx=0).stop 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 shutdown_connections 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4db406b080 0x7f4db406d540 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4dc815bdd0 0x7f4dc80ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dc80b3370 0x7f4dc80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 --2- 172.21.15.114:0/1497312387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4dc80d1c70 0x7f4dc81572d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 >> 172.21.15.114:0/1497312387 conn(0x7f4dc80731a0 msgr2=0x7f4dc815f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 shutdown_connections 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.385+0000 7f4dce070700 1 -- 172.21.15.114:0/1497312387 wait complete. 2023-11-24T06:28:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:listed 0 entries 2023-11-24T06:28:03.400 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:28:03.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:03.448 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:03.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:03.449 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:28:03.471 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:28:03.471 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:03.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:28:03.489 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:28:03.489 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:03.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:28:03.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:03.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.648+0000 7faf9f90b700 1 Processor -- start 2023-11-24T06:28:03.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.648+0000 7faf9f90b700 1 -- start start 2023-11-24T06:28:03.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9f90b700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faf98144660 0x7faf980b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9f90b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faf9806c710 0x7faf9815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9f90b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9815f520 con 0x7faf9815ea60 2023-11-24T06:28:03.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9f90b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980b4230 con 0x7faf9806c710 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9f90b700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980b3c40 con 0x7faf98144660 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faf9806c710 0x7faf9815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7faf9806c710 0x7faf9815eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42554/0 (socket says 172.21.15.114:42554) 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 -- 172.21.15.114:0/289015336 learned_addr learned my addr 172.21.15.114:0/289015336 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9d6a7700 1 --2- 172.21.15.114:0/289015336 >> v2:172.21.15.172:3300/0 conn(0x7faf98144660 0x7faf980b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 -- 172.21.15.114:0/289015336 >> v2:172.21.15.172:3300/0 conn(0x7faf98144660 msgr2=0x7faf980b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 --2- 172.21.15.114:0/289015336 >> v2:172.21.15.172:3300/0 conn(0x7faf98144660 0x7faf980b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.649+0000 7faf9dea8700 1 -- 172.21.15.114:0/289015336 >> v1:172.21.15.172:6789/0 conn(0x7faf9815ea60 legacy=0x7faf9806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9dea8700 1 -- 172.21.15.114:0/289015336 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9815f800 con 0x7faf9806c710 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9dea8700 1 --2- 172.21.15.114:0/289015336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf9806c710 0x7faf9815eef0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7faf98070670 tx=0x7faf9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca45728e0578da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.654 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf8ffff700 1 -- 172.21.15.114:0/289015336 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9000e070 con 0x7faf9806c710 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf8ffff700 1 -- 172.21.15.114:0/289015336 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf90003a20 con 0x7faf9806c710 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf8ffff700 1 -- 172.21.15.114:0/289015336 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90003dd0 con 0x7faf9806c710 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 -- 172.21.15.114:0/289015336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf9806c710 msgr2=0x7faf9815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 --2- 172.21.15.114:0/289015336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf9806c710 0x7faf9815eef0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7faf98070670 tx=0x7faf9000b330 comp rx=0 tx=0).stop 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 -- 172.21.15.114:0/289015336 shutdown_connections 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 --2- 172.21.15.114:0/289015336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf9806c710 0x7faf9815eef0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 --2- 172.21.15.114:0/289015336 >> v2:172.21.15.172:3300/0 conn(0x7faf98144660 0x7faf980b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 -- 172.21.15.114:0/289015336 >> 172.21.15.114:0/289015336 conn(0x7faf98071ac0 msgr2=0x7faf98071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 -- 172.21.15.114:0/289015336 shutdown_connections 2023-11-24T06:28:03.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.650+0000 7faf9f90b700 1 -- 172.21.15.114:0/289015336 wait complete. 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 Processor -- start 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 -- start start 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 0x7faf98159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 0x7faf9806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815d950 con 0x7faf98144660 2023-11-24T06:28:03.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815dbf0 con 0x7faf9815ea60 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9f90b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815ded0 con 0x7faf9806c710 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9cea6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9cea6700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42558/0 (socket says 172.21.15.114:42558) 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.652+0000 7faf9cea6700 1 -- 172.21.15.114:0/2694483384 learned_addr learned my addr 172.21.15.114:0/2694483384 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9d6a7700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 0x7faf98159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9dea8700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 0x7faf9806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 -- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 msgr2=0x7faf98159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 0x7faf98159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 -- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 msgr2=0x7faf9806f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 0x7faf9806f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 -- 172.21.15.114:0/2694483384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90009b30 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9d6a7700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 0x7faf98159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9dea8700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 0x7faf9806f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9cea6700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b4a0 tx=0x7faf9400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94011830 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94011990 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9815e1b0 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94011d40 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.653+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf98159f60 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.654+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf980b5690 con 0x7faf98144660 2023-11-24T06:28:03.660 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.655+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7faf9401a400 con 0x7faf98144660 2023-11-24T06:28:03.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.655+0000 7faf8dffb700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 0x7faf7c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.655+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf94089640 con 0x7faf98144660 2023-11-24T06:28:03.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.655+0000 7faf9d6a7700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 0x7faf7c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.661 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.655+0000 7faf9d6a7700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 0x7faf7c06d540 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faf8800aa50 tx=0x7faf88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.659+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf94054470 con 0x7faf98144660 2023-11-24T06:28:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.891+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf98067f60 con 0x7faf98144660 2023-11-24T06:28:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.892+0000 7faf8dffb700 1 -- 172.21.15.114:0/2694483384 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4925 (secure 0 0 0) 0x7faf94057a90 con 0x7faf98144660 2023-11-24T06:28:03.895 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:03.895 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:28:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 msgr2=0x7faf7c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 0x7faf7c06d540 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faf8800aa50 tx=0x7faf88008040 comp rx=0 tx=0).stop 2023-11-24T06:28:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 msgr2=0x7faf980afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b4a0 tx=0x7faf9400b860 comp rx=0 tx=0).stop 2023-11-24T06:28:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 shutdown_connections 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7faf7c06b080 0x7faf7c06d540 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9815ea60 0x7faf9806f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9806c710 0x7faf98159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 --2- 172.21.15.114:0/2694483384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf98144660 0x7faf980afe70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 >> 172.21.15.114:0/2694483384 conn(0x7faf98071ac0 msgr2=0x7faf980b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 shutdown_connections 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:03.894+0000 7faf9f90b700 1 -- 172.21.15.114:0/2694483384 wait complete. 2023-11-24T06:28:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2023-11-24T06:28:03.910 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:04.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.027+0000 7f8c84ed7700 1 Processor -- start 2023-11-24T06:28:04.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.027+0000 7f8c84ed7700 1 -- start start 2023-11-24T06:28:04.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c84ed7700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c800af4d0 0x7f8c800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c84ed7700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800ae430 con 0x7f8c8015e080 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c84ed7700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800ae710 con 0x7f8c8006c710 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c84ed7700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c800af0b0 con 0x7f8c800af4d0 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7e59c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8c8015e080 0x7f8c8006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58272/0 (socket says 172.21.15.114:58272) 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7e59c700 1 -- 172.21.15.114:0/3870841255 learned_addr learned my addr 172.21.15.114:0/3870841255 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3870841255 >> v2:172.21.15.172:3300/0 conn(0x7f8c800af4d0 0x7f8c800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747632803 0 0) 0x7f8c800ae430 con 0x7f8c8015e080 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c680035b0 con 0x7f8c8015e080 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3779567829 0 0) 0x7f8c680035b0 con 0x7f8c8015e080 2023-11-24T06:28:04.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 >> v2:172.21.15.172:3300/0 conn(0x7f8c800af4d0 msgr2=0x7f8c800b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 --2- 172.21.15.114:0/3870841255 >> v2:172.21.15.172:3300/0 conn(0x7f8c800af4d0 0x7f8c800b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 >> v1:172.21.15.172:6789/0 conn(0x7f8c8006c710 legacy=0x7f8c8006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800ae9f0 con 0x7f8c8015e080 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f8c70001670 con 0x7f8c8015e080 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 >> v1:172.21.15.114:6789/0 conn(0x7f8c8015e080 legacy=0x7f8c8006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c68003a00 0x7f8c68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c68004400 0x7f8c680048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 0x7f8c6800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800af0b0 con 0x7f8c680050d0 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.028+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6800c030 con 0x7f8c68004400 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6800c1f0 con 0x7f8c68003a00 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 0x7f8c6800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7e59c700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c68003a00 0x7f8c68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7dd9b700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c68004400 0x7f8c680048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 -- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c68003a00 msgr2=0x7f8c68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c68003a00 0x7f8c68003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 -- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c68004400 msgr2=0x7f8c680048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.036 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c68004400 0x7f8c680048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 -- 172.21.15.114:0/3870841255 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680035b0 con 0x7f8c680050d0 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 -- 172.21.15.114:0/3870841255 reap_dead start 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 0x7f8c6800ba10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009fc0 tx=0x7f8c68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eebd302fd2e4f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78003a20 con 0x7f8c680050d0 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800f040 con 0x7f8c680050d0 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.029+0000 7f8c7d59a700 1 -- 172.21.15.114:0/3870841255 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78003de0 con 0x7f8c680050d0 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 msgr2=0x7f8c6800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 0x7f8c6800ba10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009fc0 tx=0x7f8c68005880 comp rx=0 tx=0).stop 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3870841255 shutdown_connections 2023-11-24T06:28:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3870841255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c680050d0 0x7f8c6800ba10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3870841255 >> 172.21.15.114:0/3870841255 conn(0x7f8c80071b90 msgr2=0x7f8c80071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3870841255 shutdown_connections 2023-11-24T06:28:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.030+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3870841255 wait complete. 2023-11-24T06:28:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.031+0000 7f8c84ed7700 1 Processor -- start 2023-11-24T06:28:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.031+0000 7f8c84ed7700 1 -- start start 2023-11-24T06:28:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c680050d0 0x7f8c8015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c800cd620 0x7f8c8015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8015e460 con 0x7f8c80152830 2023-11-24T06:28:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8015ea20 con 0x7f8c680050d0 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c84ed7700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800cdfe0 con 0x7f8c800cd620 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42576/0 (socket says 172.21.15.114:42576) 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 -- 172.21.15.114:0/3490232502 learned_addr learned my addr 172.21.15.114:0/3490232502 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7ed9d700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c800cd620 0x7f8c8015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7e59c700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c680050d0 0x7f8c8015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 -- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c800cd620 msgr2=0x7f8c8015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c800cd620 0x7f8c8015cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 -- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c680050d0 msgr2=0x7f8c8015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c680050d0 0x7f8c8015eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.043 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 -- 172.21.15.114:0/3490232502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800ce2c0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.032+0000 7f8c7dd9b700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400f7b0 tx=0x7f8c7400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740052e0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74005460 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8015e6b0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740038c0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c800cdab0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.033+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c800753f0 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.034+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8c74003a20 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.034+0000 7f8c6effd700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 0x7f8c6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.034+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c74089770 con 0x7f8c80152830 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.034+0000 7f8c7e59c700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 0x7f8c6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.035+0000 7f8c7e59c700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 0x7f8c6006d540 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c70001050 tx=0x7f8c70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.045 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.039+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c74054630 con 0x7f8c80152830 2023-11-24T06:28:04.235 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.233+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c80067f60 con 0x7f8c80152830 2023-11-24T06:28:04.236 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.234+0000 7f8c6effd700 1 -- 172.21.15.114:0/3490232502 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f8c74057c50 con 0x7f8c80152830 2023-11-24T06:28:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:04.237 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:04.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 msgr2=0x7f8c6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.238 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 0x7f8c6006d540 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c70001050 tx=0x7f8c70009040 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 msgr2=0x7f8c801519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400f7b0 tx=0x7f8c7400fb70 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 shutdown_connections 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8c6006b080 0x7f8c6006d540 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c800cd620 0x7f8c8015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c680050d0 0x7f8c8015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 --2- 172.21.15.114:0/3490232502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c80152830 0x7f8c801519b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 >> 172.21.15.114:0/3490232502 conn(0x7f8c80071b90 msgr2=0x7f8c800b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 shutdown_connections 2023-11-24T06:28:04.239 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.236+0000 7f8c84ed7700 1 -- 172.21.15.114:0/3490232502 wait complete. 2023-11-24T06:28:04.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:24:05.750313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:27:41.963773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:27:43.243874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:04.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:04.374 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.372+0000 7f98ba90a700 1 Processor -- start 2023-11-24T06:28:04.375 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.372+0000 7f98ba90a700 1 -- start start 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98ba90a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f98b415c0c0 0x7f98b40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98ba90a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b40d2000 0x7f98b40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98ba90a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40d04d0 con 0x7f98b40cfa80 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98ba90a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40d07b0 con 0x7f98b415c0c0 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98ba90a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40d0a90 con 0x7f98b40d2000 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f98b415c0c0 0x7f98b40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f98b415c0c0 0x7f98b40b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42584/0 (socket says 172.21.15.114:42584) 2023-11-24T06:28:04.377 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 -- 172.21.15.114:0/2997203138 learned_addr learned my addr 172.21.15.114:0/2997203138 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b8ea7700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b40d2000 0x7f98b40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 -- 172.21.15.114:0/2997203138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b40d2000 msgr2=0x7f98b40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b40d2000 0x7f98b40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 -- 172.21.15.114:0/2997203138 >> v1:172.21.15.172:6789/0 conn(0x7f98b40cfa80 legacy=0x7f98b40d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.373+0000 7f98b3fff700 1 -- 172.21.15.114:0/2997203138 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40ccb30 con 0x7f98b415c0c0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98b3fff700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b415c0c0 0x7f98b40b1ad0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f98a80098f0 tx=0x7f98a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91b502abbee9c9e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98b2ffd700 1 -- 172.21.15.114:0/2997203138 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004230 con 0x7f98b415c0c0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98b2ffd700 1 -- 172.21.15.114:0/2997203138 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8004390 con 0x7f98b415c0c0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98b2ffd700 1 -- 172.21.15.114:0/2997203138 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80036a0 con 0x7f98b415c0c0 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 -- 172.21.15.114:0/2997203138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b415c0c0 msgr2=0x7f98b40b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b415c0c0 0x7f98b40b1ad0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f98a80098f0 tx=0x7f98a8009c00 comp rx=0 tx=0).stop 2023-11-24T06:28:04.378 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 -- 172.21.15.114:0/2997203138 shutdown_connections 2023-11-24T06:28:04.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b40d2000 0x7f98b40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 --2- 172.21.15.114:0/2997203138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b415c0c0 0x7f98b40b1ad0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 -- 172.21.15.114:0/2997203138 >> 172.21.15.114:0/2997203138 conn(0x7f98b409aa00 msgr2=0x7f98b409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 -- 172.21.15.114:0/2997203138 shutdown_connections 2023-11-24T06:28:04.379 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.374+0000 7f98ba90a700 1 -- 172.21.15.114:0/2997203138 wait complete. 2023-11-24T06:28:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 Processor -- start 2023-11-24T06:28:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 -- start start 2023-11-24T06:28:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b40cfa80 0x7f98b415f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 0x7f98b414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b415c6c0 con 0x7f98b40d2000 2023-11-24T06:28:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40b6e40 con 0x7f98b40cfa80 2023-11-24T06:28:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.376+0000 7f98ba90a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40b7120 con 0x7f98b415c0c0 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42590/0 (socket says 172.21.15.114:42590) 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 -- 172.21.15.114:0/989552084 learned_addr learned my addr 172.21.15.114:0/989552084 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b8ea7700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 0x7f98b414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b3fff700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b40cfa80 0x7f98b415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 -- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 msgr2=0x7f98b414ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.388 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 0x7f98b414ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 -- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b40cfa80 msgr2=0x7f98b415f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.389 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b40cfa80 0x7f98b415f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.390 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 -- 172.21.15.114:0/989552084 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8009550 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b8ea7700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 0x7f98b414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b37fe700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b4a0 tx=0x7f98a400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4011830 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4011990 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b40b7400 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4011d40 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.377+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b414f390 con 0x7f98b40d2000 2023-11-24T06:28:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.378+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b415c830 con 0x7f98b40d2000 2023-11-24T06:28:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.379+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f98a401a400 con 0x7f98b40d2000 2023-11-24T06:28:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.379+0000 7f98b0ff9700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 0x7f98a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.379+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98a40895b0 con 0x7f98b40d2000 2023-11-24T06:28:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.379+0000 7f98b3fff700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 0x7f98a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.379+0000 7f98b3fff700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 0x7f98a006d540 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009130 tx=0x7f98a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.393 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.383+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98b415c830 con 0x7f98b40d2000 2023-11-24T06:28:04.574 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.572+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98b415c800 con 0x7f98b40d2000 2023-11-24T06:28:04.575 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.573+0000 7f98b0ff9700 1 -- 172.21.15.114:0/989552084 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98a4054470 con 0x7f98b40d2000 2023-11-24T06:28:04.575 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 msgr2=0x7f98a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 0x7f98a006d540 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009130 tx=0x7f98a8008030 comp rx=0 tx=0).stop 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 msgr2=0x7f98b40b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b4a0 tx=0x7f98a400b860 comp rx=0 tx=0).stop 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 shutdown_connections 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f98a006b080 0x7f98a006d540 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98b415c0c0 0x7f98b414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98b40cfa80 0x7f98b415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 --2- 172.21.15.114:0/989552084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98b40d2000 0x7f98b40b47a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 >> 172.21.15.114:0/989552084 conn(0x7f98b409aa00 msgr2=0x7f98b40cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 shutdown_connections 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.575+0000 7f98ba90a700 1 -- 172.21.15.114:0/989552084 wait complete. 2023-11-24T06:28:04.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:04.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.711+0000 7f1c83bb9700 1 Processor -- start 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.711+0000 7f1c83bb9700 1 -- start start 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c83bb9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c7c15e160 0x7f1c7c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c83bb9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c7c0746a0 0x7f1c7c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c83bb9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c0b3c60 con 0x7f1c7c0afc90 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c83bb9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b4500 con 0x7f1c7c15e160 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c83bb9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b58c0 con 0x7f1c7c0746a0 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c7c15e160 0x7f1c7c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c7c15e160 0x7f1c7c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42596/0 (socket says 172.21.15.114:42596) 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 -- 172.21.15.114:0/3601595492 learned_addr learned my addr 172.21.15.114:0/3601595492 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:04.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 -- 172.21.15.114:0/3601595492 >> v2:172.21.15.172:3300/0 conn(0x7f1c7c0746a0 msgr2=0x7f1c7c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 --2- 172.21.15.114:0/3601595492 >> v2:172.21.15.172:3300/0 conn(0x7f1c7c0746a0 0x7f1c7c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 -- 172.21.15.114:0/3601595492 >> v1:172.21.15.172:6789/0 conn(0x7f1c7c0afc90 legacy=0x7f1c7c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 -- 172.21.15.114:0/3601595492 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0b0140 con 0x7f1c7c15e160 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c81955700 1 --2- 172.21.15.114:0/3601595492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c15e160 0x7f1c7c0b5da0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0d16e0 tx=0x7f1c78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e5e4210434908 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c80953700 1 -- 172.21.15.114:0/3601595492 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c78004230 con 0x7f1c7c15e160 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c80953700 1 -- 172.21.15.114:0/3601595492 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c78004390 con 0x7f1c7c15e160 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.712+0000 7f1c80953700 1 -- 172.21.15.114:0/3601595492 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c780036a0 con 0x7f1c7c15e160 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 -- 172.21.15.114:0/3601595492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c15e160 msgr2=0x7f1c7c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/3601595492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c15e160 0x7f1c7c0b5da0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0d16e0 tx=0x7f1c78009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 -- 172.21.15.114:0/3601595492 shutdown_connections 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/3601595492 >> v2:172.21.15.172:3300/0 conn(0x7f1c7c0746a0 0x7f1c7c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.717 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/3601595492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c15e160 0x7f1c7c0b5da0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 -- 172.21.15.114:0/3601595492 >> 172.21.15.114:0/3601595492 conn(0x7f1c7c0d0bf0 msgr2=0x7f1c7c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 -- 172.21.15.114:0/3601595492 shutdown_connections 2023-11-24T06:28:04.718 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.713+0000 7f1c83bb9700 1 -- 172.21.15.114:0/3601595492 wait complete. 2023-11-24T06:28:04.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.714+0000 7f1c83bb9700 1 Processor -- start 2023-11-24T06:28:04.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.714+0000 7f1c83bb9700 1 -- start start 2023-11-24T06:28:04.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.721 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7c15e160 0x7f1c7c1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15e880 con 0x7f1c7c0afc90 2023-11-24T06:28:04.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15eb60 con 0x7f1c7c0746a0 2023-11-24T06:28:04.722 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c83bb9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15ee40 con 0x7f1c7c15e160 2023-11-24T06:28:04.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42604/0 (socket says 172.21.15.114:42604) 2023-11-24T06:28:04.723 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 -- 172.21.15.114:0/2763434607 learned_addr learned my addr 172.21.15.114:0/2763434607 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:04.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81955700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 -- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7c15e160 msgr2=0x7f1c7c1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7c15e160 0x7f1c7c1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.724 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 -- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7c0746a0 msgr2=0x7f1c7c154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 -- 172.21.15.114:0/2763434607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78009550 con 0x7f1c7c0afc90 2023-11-24T06:28:04.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.715+0000 7f1c81154700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c4a0 tx=0x7f1c6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.716+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0054e0 con 0x7f1c7c0afc90 2023-11-24T06:28:04.725 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.716+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c003680 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.716+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c15f120 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.716+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003a30 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.716+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c0c7880 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.717+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c068fb0 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.717+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f1c6c003b90 con 0x7f1c7c0afc90 2023-11-24T06:28:04.726 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.717+0000 7f1c727fc700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 0x7f1c6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.717+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c6c088dd0 con 0x7f1c7c0afc90 2023-11-24T06:28:04.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.718+0000 7f1c81955700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 0x7f1c6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.718+0000 7f1c81955700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 0x7f1c6006d540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009c10 tx=0x7f1c78008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.727 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.722+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c6c053c90 con 0x7f1c7c0afc90 2023-11-24T06:28:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.913+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c7c0b46b0 con 0x7f1c7c0afc90 2023-11-24T06:28:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.913+0000 7f1c727fc700 1 -- 172.21.15.114:0/2763434607 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c6c08a020 con 0x7f1c7c0afc90 2023-11-24T06:28:04.916 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2023-11-24T06:28:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 msgr2=0x7f1c6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 0x7f1c6006d540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009c10 tx=0x7f1c78008030 comp rx=0 tx=0).stop 2023-11-24T06:28:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 msgr2=0x7f1c7c1550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c4a0 tx=0x7f1c6c00c860 comp rx=0 tx=0).stop 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 shutdown_connections 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f1c6006b080 0x7f1c6006d540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c7c15e160 0x7f1c7c1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 --2- 172.21.15.114:0/2763434607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c1550f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 >> 172.21.15.114:0/2763434607 conn(0x7f1c7c0d0bf0 msgr2=0x7f1c7c15d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 shutdown_connections 2023-11-24T06:28:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:04.915+0000 7f1c83bb9700 1 -- 172.21.15.114:0/2763434607 wait complete. 2023-11-24T06:28:04.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:05.060 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.050+0000 7f6231328700 1 Processor -- start 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.050+0000 7f6231328700 1 -- start start 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6231328700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f622c0d45d0 0x7f622c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6231328700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f622c0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6231328700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c0d2930 con 0x7f622c15e140 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6231328700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c15e520 con 0x7f622c0d45d0 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6231328700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0d2c10 con 0x7f622c0d39d0 2023-11-24T06:28:05.061 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f622affd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f622c15e140 0x7f622c0d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58318/0 (socket says 172.21.15.114:58318) 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f622affd700 1 -- 172.21.15.114:0/3493863239 learned_addr learned my addr 172.21.15.114:0/3493863239 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008806720 0 0) 0x7f622c0d2930 con 0x7f622c15e140 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f622a7fc700 1 --2- 172.21.15.114:0/3493863239 >> v2:172.21.15.172:3300/0 conn(0x7f622c0d45d0 0x7f622c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62140035b0 con 0x7f622c15e140 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f622b7fe700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f622c0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 442569496 0 0) 0x7f62140035b0 con 0x7f622c15e140 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 msgr2=0x7f622c0b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f622c0b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 >> v2:172.21.15.172:3300/0 conn(0x7f622c0d45d0 msgr2=0x7f622c0b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 --2- 172.21.15.114:0/3493863239 >> v2:172.21.15.172:3300/0 conn(0x7f622c0d45d0 0x7f622c0b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0d2ef0 con 0x7f622c15e140 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f6220001670 con 0x7f622c15e140 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 >> v1:172.21.15.114:6789/0 conn(0x7f622c15e140 legacy=0x7f622c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6214003a00 0x7f6214003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f6214004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f6214005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c15e520 con 0x7f6214004850 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0d2c10 con 0x7f6214003a00 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621400ba10 con 0x7f622c0d39d0 2023-11-24T06:28:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.051+0000 7f622a7fc700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f6214004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622affd700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6214003a00 0x7f6214003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622b7fe700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f6214005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 -- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 msgr2=0x7f6214005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f6214005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 -- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6214003a00 msgr2=0x7f6214003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6214003a00 0x7f6214003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 -- 172.21.15.114:0/3493863239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62140035b0 con 0x7f6214004850 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622affd700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6214003a00 0x7f6214003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622b7fe700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0d39d0 0x7f6214005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622a7fc700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f6214004cd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f621c009b60 tx=0x7f6214005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c977569acdd3774d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00e660 con 0x7f6214004850 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c00e7c0 con 0x7f6214004850 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f6229ffb700 1 -- 172.21.15.114:0/3493863239 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00eb30 con 0x7f6214004850 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f6231328700 1 -- 172.21.15.114:0/3493863239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 msgr2=0x7f6214004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f6231328700 1 --2- 172.21.15.114:0/3493863239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f6214004cd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f621c009b60 tx=0x7f6214005880 comp rx=0 tx=0).stop 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.052+0000 7f622b7fe700 1 -- 172.21.15.114:0/3493863239 reap_dead start 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.053+0000 7f6231328700 1 -- 172.21.15.114:0/3493863239 shutdown_connections 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.053+0000 7f6231328700 1 -- 172.21.15.114:0/3493863239 >> 172.21.15.114:0/3493863239 conn(0x7f622c06b410 msgr2=0x7f622c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.053+0000 7f6231328700 1 -- 172.21.15.114:0/3493863239 shutdown_connections 2023-11-24T06:28:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.053+0000 7f6231328700 1 -- 172.21.15.114:0/3493863239 wait complete. 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.054+0000 7f6231328700 1 Processor -- start 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.054+0000 7f6231328700 1 -- start start 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0b3970 0x7f622c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c0ccf10 0x7f622c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0b4330 con 0x7f6214004850 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0c7c10 con 0x7f622c0ccf10 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0c7ef0 con 0x7f622c0b3970 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42622/0 (socket says 172.21.15.114:42622) 2023-11-24T06:28:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 -- 172.21.15.114:0/458890913 learned_addr learned my addr 172.21.15.114:0/458890913 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622a7fc700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0b3970 0x7f622c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622b7fe700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c0ccf10 0x7f622c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 -- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0b3970 msgr2=0x7f622c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0b3970 0x7f622c0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 -- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c0ccf10 msgr2=0x7f622c0cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c0ccf10 0x7f622c0cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 -- 172.21.15.114:0/458890913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0c81d0 con 0x7f6214004850 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f622affd700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6220000fb0 tx=0x7f622000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220005e20 con 0x7f6214004850 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622000c780 con 0x7f6214004850 2023-11-24T06:28:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c0c84b0 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000cb10 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.055+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c14a7e0 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.056+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c0c8b00 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.057+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f622000cc70 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.057+0000 7f62137fe700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 0x7f621806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.057+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6220089860 con 0x7f6214004850 2023-11-24T06:28:05.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.057+0000 7f622a7fc700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 0x7f621806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.057+0000 7f622a7fc700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 0x7f621806d540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f621c0090b0 tx=0x7f621c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.061+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6220054690 con 0x7f6214004850 2023-11-24T06:28:05.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.250+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f622c0d1bb0 con 0x7f6214004850 2023-11-24T06:28:05.253 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.251+0000 7f62137fe700 1 -- 172.21.15.114:0/458890913 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6220057cb0 con 0x7f6214004850 2023-11-24T06:28:05.253 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2023-11-24T06:28:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 msgr2=0x7f621806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 0x7f621806d540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f621c0090b0 tx=0x7f621c008030 comp rx=0 tx=0).stop 2023-11-24T06:28:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 msgr2=0x7f622c14a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.255 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6220000fb0 tx=0x7f622000ee80 comp rx=0 tx=0).stop 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 shutdown_connections 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f621806b080 0x7f621806d540 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f622c0ccf10 0x7f622c0cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6214004850 0x7f622c14a2a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 --2- 172.21.15.114:0/458890913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f622c0b3970 0x7f622c0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 >> 172.21.15.114:0/458890913 conn(0x7f622c06b410 msgr2=0x7f622c15eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 shutdown_connections 2023-11-24T06:28:05.256 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.253+0000 7f6231328700 1 -- 172.21.15.114:0/458890913 wait complete. 2023-11-24T06:28:05.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.386+0000 7f6dc008e700 1 Processor -- start 2023-11-24T06:28:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.386+0000 7f6dc008e700 1 -- start start 2023-11-24T06:28:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 0x7f6db80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6db80af4d0 0x7f6db80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6db806c000 0x7f6db806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db80af0b0 con 0x7f6db806c000 2023-11-24T06:28:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db806c9f0 con 0x7f6db80af4d0 2023-11-24T06:28:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dc008e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80b05a0 con 0x7f6db815e080 2023-11-24T06:28:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6db806c000 0x7f6db806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6db806c000 0x7f6db806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42634/0 (socket says 172.21.15.114:42634) 2023-11-24T06:28:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 -- 172.21.15.114:0/8866694 learned_addr learned my addr 172.21.15.114:0/8866694 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:05.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 0x7f6db80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbd629700 1 --2- 172.21.15.114:0/8866694 >> v2:172.21.15.172:3300/0 conn(0x7f6db80af4d0 0x7f6db80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.409 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 -- 172.21.15.114:0/8866694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 msgr2=0x7f6db80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 0x7f6db80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 -- 172.21.15.114:0/8866694 >> v2:172.21.15.172:3300/0 conn(0x7f6db80af4d0 msgr2=0x7f6db80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.387+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/8866694 >> v2:172.21.15.172:3300/0 conn(0x7f6db80af4d0 0x7f6db80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.410 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbe62b700 1 -- 172.21.15.114:0/8866694 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80ae430 con 0x7f6db806c000 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 0x7f6db80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db806c000 0x7f6db806c4b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6db806d380 tx=0x7f6db000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceccbb0a8b0f116 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbce28700 1 -- 172.21.15.114:0/8866694 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000e070 con 0x7f6db806c000 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbce28700 1 -- 172.21.15.114:0/8866694 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db000b980 con 0x7f6db806c000 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dbce28700 1 -- 172.21.15.114:0/8866694 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000bd30 con 0x7f6db806c000 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 -- 172.21.15.114:0/8866694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db806c000 msgr2=0x7f6db806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db806c000 0x7f6db806c4b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6db806d380 tx=0x7f6db000b330 comp rx=0 tx=0).stop 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 -- 172.21.15.114:0/8866694 shutdown_connections 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db815e080 0x7f6db80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 --2- 172.21.15.114:0/8866694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db806c000 0x7f6db806c4b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 --2- 172.21.15.114:0/8866694 >> v2:172.21.15.172:3300/0 conn(0x7f6db80af4d0 0x7f6db80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.388+0000 7f6dc008e700 1 -- 172.21.15.114:0/8866694 >> 172.21.15.114:0/8866694 conn(0x7f6db8071b90 msgr2=0x7f6db8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.412 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.389+0000 7f6dc008e700 1 -- 172.21.15.114:0/8866694 shutdown_connections 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.389+0000 7f6dc008e700 1 -- 172.21.15.114:0/8866694 wait complete. 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 Processor -- start 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 -- start start 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 0x7f6db81f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db80af4d0 0x7f6db81f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.413 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81f7960 con 0x7f6db815e080 2023-11-24T06:28:05.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db8144460 con 0x7f6db806c000 2023-11-24T06:28:05.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.390+0000 7f6dc008e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db8141d40 con 0x7f6db80af4d0 2023-11-24T06:28:05.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.414 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42640/0 (socket says 172.21.15.114:42640) 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 -- 172.21.15.114:0/807941061 learned_addr learned my addr 172.21.15.114:0/807941061 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbd629700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db80af4d0 0x7f6db81f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 0x7f6db81f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 -- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db80af4d0 msgr2=0x7f6db81f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db80af4d0 0x7f6db81f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 -- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 msgr2=0x7f6db81f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 0x7f6db81f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 -- 172.21.15.114:0/807941061 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009b30 con 0x7f6db815e080 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 0x7f6db81f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dbe62b700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6db809f040 tx=0x7f6db0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004250 con 0x7f6db815e080 2023-11-24T06:28:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00043b0 con 0x7f6db815e080 2023-11-24T06:28:05.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db8144130 con 0x7f6db815e080 2023-11-24T06:28:05.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004760 con 0x7f6db815e080 2023-11-24T06:28:05.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.391+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db8160fc0 con 0x7f6db815e080 2023-11-24T06:28:05.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.392+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db80753f0 con 0x7f6db815e080 2023-11-24T06:28:05.417 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.393+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6db001c440 con 0x7f6db815e080 2023-11-24T06:28:05.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.393+0000 7f6dae7fc700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 0x7f6d9c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.393+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 0x7f6d9c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.393+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6db008b270 con 0x7f6db815e080 2023-11-24T06:28:05.418 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.393+0000 7f6dbde2a700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 0x7f6d9c06d540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6db400aa20 tx=0x7f6db4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.419 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.398+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6db00560c0 con 0x7f6db815e080 2023-11-24T06:28:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.631+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db8067f60 con 0x7f6db815e080 2023-11-24T06:28:05.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.631+0000 7f6dae7fc700 1 -- 172.21.15.114:0/807941061 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f6db00596e0 con 0x7f6db815e080 2023-11-24T06:28:05.635 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:05.636 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:28:05.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 msgr2=0x7f6d9c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 0x7f6d9c06d540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6db400aa20 tx=0x7f6db4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:05.638 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 msgr2=0x7f6db81448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6db809f040 tx=0x7f6db0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 shutdown_connections 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6d9c06b080 0x7f6d9c06d540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6db815e080 0x7f6db81448f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6db806c000 0x7f6db81f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 --2- 172.21.15.114:0/807941061 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6db80af4d0 0x7f6db81f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.639 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 >> 172.21.15.114:0/807941061 conn(0x7f6db8071b90 msgr2=0x7f6db815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 shutdown_connections 2023-11-24T06:28:05.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.633+0000 7f6dc008e700 1 -- 172.21.15.114:0/807941061 wait complete. 2023-11-24T06:28:05.640 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2023-11-24T06:28:05.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:05.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.764+0000 7fa78fafc700 1 Processor -- start 2023-11-24T06:28:05.767 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 -- start start 2023-11-24T06:28:05.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7881446a0 0x7fa7880b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.769 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa78806c710 0x7fa7880b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880af950 con 0x7fa7880adf00 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa788144a80 con 0x7fa7881446a0 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.765+0000 7fa78fafc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880afc30 con 0x7fa78806c710 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7881446a0 0x7fa7880b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d097700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa7880adf00 0x7fa78806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58354/0 (socket says 172.21.15.114:58354) 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7881446a0 0x7fa7880b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42644/0 (socket says 172.21.15.114:42644) 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d097700 1 -- 172.21.15.114:0/3718708590 learned_addr learned my addr 172.21.15.114:0/3718708590 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 -- 172.21.15.114:0/3718708590 >> v2:172.21.15.172:3300/0 conn(0x7fa78806c710 msgr2=0x7fa7880b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78e099700 1 --2- 172.21.15.114:0/3718708590 >> v2:172.21.15.172:3300/0 conn(0x7fa78806c710 0x7fa7880b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 --2- 172.21.15.114:0/3718708590 >> v2:172.21.15.172:3300/0 conn(0x7fa78806c710 0x7fa7880b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 -- 172.21.15.114:0/3718708590 >> v1:172.21.15.114:6789/0 conn(0x7fa7880adf00 legacy=0x7fa78806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 -- 172.21.15.114:0/3718708590 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880aff10 con 0x7fa7881446a0 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78c896700 1 -- 172.21.15.114:0/3718708590 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237956622 0 0) 0x7fa7880af950 con 0x7fa7880adf00 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78d898700 1 --2- 172.21.15.114:0/3718708590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7881446a0 0x7fa7880b5ae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa7880aedf0 tx=0x7fa77c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c33d9c22f6de57e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78c896700 1 -- 172.21.15.114:0/3718708590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c011830 con 0x7fa7881446a0 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78c896700 1 -- 172.21.15.114:0/3718708590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c011990 con 0x7fa7881446a0 2023-11-24T06:28:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.766+0000 7fa78c896700 1 -- 172.21.15.114:0/3718708590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c011d20 con 0x7fa7881446a0 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 -- 172.21.15.114:0/3718708590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7881446a0 msgr2=0x7fa7880b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3718708590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7881446a0 0x7fa7880b5ae0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa7880aedf0 tx=0x7fa77c009b20 comp rx=0 tx=0).stop 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 -- 172.21.15.114:0/3718708590 shutdown_connections 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3718708590 >> v2:172.21.15.172:3300/0 conn(0x7fa78806c710 0x7fa7880b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3718708590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7881446a0 0x7fa7880b5ae0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 -- 172.21.15.114:0/3718708590 >> 172.21.15.114:0/3718708590 conn(0x7fa788071b90 msgr2=0x7fa788071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 -- 172.21.15.114:0/3718708590 shutdown_connections 2023-11-24T06:28:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.767+0000 7fa78fafc700 1 -- 172.21.15.114:0/3718708590 wait complete. 2023-11-24T06:28:05.775 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.768+0000 7fa78fafc700 1 Processor -- start 2023-11-24T06:28:05.775 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.768+0000 7fa78fafc700 1 -- start start 2023-11-24T06:28:05.775 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78806c710 0x7fa788158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.775 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7881446a0 0x7fa788158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78809b030 con 0x7fa7880adf00 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78809aa00 con 0x7fa7881446a0 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78fafc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78815abd0 con 0x7fa78806c710 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78d097700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78d097700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42652/0 (socket says 172.21.15.114:42652) 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.769+0000 7fa78d097700 1 -- 172.21.15.114:0/3515218865 learned_addr learned my addr 172.21.15.114:0/3515218865 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78e099700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7881446a0 0x7fa788158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 -- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78806c710 msgr2=0x7fa788158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78806c710 0x7fa788158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 -- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7881446a0 msgr2=0x7fa788158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7881446a0 0x7fa788158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 -- 172.21.15.114:0/3515218865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0094a0 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78d097700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa78400c500 tx=0x7fa78400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784004c80 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784004e00 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7881439d0 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784007310 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.770+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78809ac50 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.771+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7880753f0 con 0x7fa7880adf00 2023-11-24T06:28:05.777 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.771+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa784007470 con 0x7fa7880adf00 2023-11-24T06:28:05.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.772+0000 7fa77a7fc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 0x7fa76c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.772+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa784089620 con 0x7fa7880adf00 2023-11-24T06:28:05.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.772+0000 7fa78d898700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 0x7fa76c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.772+0000 7fa78d898700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 0x7fa76c06d540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa7880ccef0 tx=0x7fa77c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.778 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.776+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa784054450 con 0x7fa7880adf00 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.967+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa788067f60 con 0x7fa7880adf00 2023-11-24T06:28:05.971 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.968+0000 7fa77a7fc700 1 -- 172.21.15.114:0/3515218865 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fa784057a70 con 0x7fa7880adf00 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 msgr2=0x7fa76c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 0x7fa76c06d540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa7880ccef0 tx=0x7fa77c010000 comp rx=0 tx=0).stop 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 msgr2=0x7fa7880cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa78400c500 tx=0x7fa78400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 shutdown_connections 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fa76c06b080 0x7fa76c06d540 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7881446a0 0x7fa788158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78806c710 0x7fa788158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 --2- 172.21.15.114:0/3515218865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7880adf00 0x7fa7880cffc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.970+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 >> 172.21.15.114:0/3515218865 conn(0x7fa788071b90 msgr2=0x7fa78815d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.971+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 shutdown_connections 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:05.971+0000 7fa78fafc700 1 -- 172.21.15.114:0/3515218865 wait complete. 2023-11-24T06:28:05.986 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:28:05.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:06.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.104+0000 7f50aa0ec700 1 Processor -- start 2023-11-24T06:28:06.106 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.104+0000 7f50aa0ec700 1 -- start start 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50aa0ec700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f50a415dfc0 0x7f50a40d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50aa0ec700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 0x7f50a40d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50aa0ec700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a40d2d10 con 0x7f50a40b2a70 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50aa0ec700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a40cefd0 con 0x7f50a415dfc0 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50aa0ec700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a40cf2b0 con 0x7f50a40b2fc0 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f50a415dfc0 0x7f50a40d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f50a415dfc0 0x7f50a40d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42658/0 (socket says 172.21.15.114:42658) 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 -- 172.21.15.114:0/2339675388 learned_addr learned my addr 172.21.15.114:0/2339675388 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a2ffd700 1 --2- 172.21.15.114:0/2339675388 >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 0x7f50a40d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 -- 172.21.15.114:0/2339675388 >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 msgr2=0x7f50a40d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.109 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 --2- 172.21.15.114:0/2339675388 >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 0x7f50a40d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 -- 172.21.15.114:0/2339675388 >> v1:172.21.15.172:6789/0 conn(0x7f50a40b2a70 legacy=0x7f50a4161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 -- 172.21.15.114:0/2339675388 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40d2fb0 con 0x7f50a415dfc0 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a2ffd700 1 --2- 172.21.15.114:0/2339675388 >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 0x7f50a40d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a37fe700 1 --2- 172.21.15.114:0/2339675388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a415dfc0 0x7f50a40d0e60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f50a40b5f70 tx=0x7f508c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a406b026d7bec3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.105+0000 7f50a27fc700 1 -- 172.21.15.114:0/2339675388 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00b870 con 0x7f50a415dfc0 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50a27fc700 1 -- 172.21.15.114:0/2339675388 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00b9d0 con 0x7f50a415dfc0 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50a27fc700 1 -- 172.21.15.114:0/2339675388 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00bd60 con 0x7f50a415dfc0 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 -- 172.21.15.114:0/2339675388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a415dfc0 msgr2=0x7f50a40d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/2339675388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a415dfc0 0x7f50a40d0e60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f50a40b5f70 tx=0x7f508c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 -- 172.21.15.114:0/2339675388 shutdown_connections 2023-11-24T06:28:06.110 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/2339675388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a415dfc0 0x7f50a40d0e60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/2339675388 >> v2:172.21.15.172:3300/0 conn(0x7f50a40b2fc0 0x7f50a40d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 -- 172.21.15.114:0/2339675388 >> 172.21.15.114:0/2339675388 conn(0x7f50a406b410 msgr2=0x7f50a406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 -- 172.21.15.114:0/2339675388 shutdown_connections 2023-11-24T06:28:06.111 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.106+0000 7f50aa0ec700 1 -- 172.21.15.114:0/2339675388 wait complete. 2023-11-24T06:28:06.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.107+0000 7f50aa0ec700 1 Processor -- start 2023-11-24T06:28:06.113 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 -- start start 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 0x7f50a40b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 0x7f50a414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a415c7e0 con 0x7f50a40b2fc0 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a414dfb0 con 0x7f50a40b2a70 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50aa0ec700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a414e290 con 0x7f50a415dfc0 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50a2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50a2ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42666/0 (socket says 172.21.15.114:42666) 2023-11-24T06:28:06.114 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.108+0000 7f50a2ffd700 1 -- 172.21.15.114:0/1820845897 learned_addr learned my addr 172.21.15.114:0/1820845897 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a37fe700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 0x7f50a40b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a3fff700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 0x7f50a414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 -- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 msgr2=0x7f50a414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 0x7f50a414d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 -- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 msgr2=0x7f50a40b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 0x7f50a40b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 -- 172.21.15.114:0/1820845897 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009550 con 0x7f50a40b2fc0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a37fe700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 0x7f50a40b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a3fff700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 0x7f50a414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50a2ffd700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f509400e810 tx=0x7f509400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094013610 con 0x7f50a40b2fc0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094013770 con 0x7f50a40b2fc0 2023-11-24T06:28:06.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a414e570 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094013b20 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.109+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a414eb30 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.110+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4067f60 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.110+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5094013c80 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.111+0000 7f509bfff700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 0x7f508406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.111+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5094089650 con 0x7f50a40b2fc0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.111+0000 7f50a37fe700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 0x7f508406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.116 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.111+0000 7f50a37fe700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 0x7f508406d540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f508c009130 tx=0x7f508c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.115+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5094054400 con 0x7f50a40b2fc0 2023-11-24T06:28:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.348+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50a40ced10 con 0x7f50a40b2fc0 2023-11-24T06:28:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.349+0000 7f509bfff700 1 -- 172.21.15.114:0/1820845897 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f50940094b0 con 0x7f50a40b2fc0 2023-11-24T06:28:06.351 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:06.352 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:28:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 msgr2=0x7f508406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 0x7f508406d540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f508c009130 tx=0x7f508c008030 comp rx=0 tx=0).stop 2023-11-24T06:28:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 msgr2=0x7f50a414d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f509400e810 tx=0x7f509400ebd0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 shutdown_connections 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f508406b080 0x7f508406d540 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50a415dfc0 0x7f50a414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50a40b2a70 0x7f50a40b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 --2- 172.21.15.114:0/1820845897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a40b2fc0 0x7f50a414d440 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 >> 172.21.15.114:0/1820845897 conn(0x7f50a406b410 msgr2=0x7f50a40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 shutdown_connections 2023-11-24T06:28:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.351+0000 7f50aa0ec700 1 -- 172.21.15.114:0/1820845897 wait complete. 2023-11-24T06:28:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2023-11-24T06:28:06.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.483+0000 7f0d81adc700 1 Processor -- start 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.483+0000 7f0d81adc700 1 -- start start 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c15e080 0x7f0d7c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0d7c0af4d0 0x7f0d7c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d7c06c000 0x7f0d7c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0af0b0 con 0x7f0d7c06c000 2023-11-24T06:28:06.488 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c06c9f0 con 0x7f0d7c0af4d0 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d81adc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0b05a0 con 0x7f0d7c15e080 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d7c06c000 0x7f0d7c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d7c06c000 0x7f0d7c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42676/0 (socket says 172.21.15.114:42676) 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 -- 172.21.15.114:0/2165121602 learned_addr learned my addr 172.21.15.114:0/2165121602 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7b7fe700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c15e080 0x7f0d7c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7affd700 1 --2- 172.21.15.114:0/2165121602 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c0af4d0 0x7f0d7c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 -- 172.21.15.114:0/2165121602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c15e080 msgr2=0x7f0d7c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.489 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c15e080 0x7f0d7c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 -- 172.21.15.114:0/2165121602 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c0af4d0 msgr2=0x7f0d7c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 --2- 172.21.15.114:0/2165121602 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c0af4d0 0x7f0d7c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 -- 172.21.15.114:0/2165121602 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0ae430 con 0x7f0d7c06c000 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7bfff700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c06c4b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c06d380 tx=0x7f0d7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282cfa3aa10cf0eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7a7fc700 1 -- 172.21.15.114:0/2165121602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400e070 con 0x7f0d7c06c000 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7a7fc700 1 -- 172.21.15.114:0/2165121602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400b980 con 0x7f0d7c06c000 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.484+0000 7f0d7a7fc700 1 -- 172.21.15.114:0/2165121602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400bd30 con 0x7f0d7c06c000 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 -- 172.21.15.114:0/2165121602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 msgr2=0x7f0d7c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.490 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c06c4b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c06d380 tx=0x7f0d7400b330 comp rx=0 tx=0).stop 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 -- 172.21.15.114:0/2165121602 shutdown_connections 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c15e080 0x7f0d7c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 --2- 172.21.15.114:0/2165121602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c06c4b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 --2- 172.21.15.114:0/2165121602 >> v2:172.21.15.172:3300/0 conn(0x7f0d7c0af4d0 0x7f0d7c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 -- 172.21.15.114:0/2165121602 >> 172.21.15.114:0/2165121602 conn(0x7f0d7c071b90 msgr2=0x7f0d7c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.491 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 -- 172.21.15.114:0/2165121602 shutdown_connections 2023-11-24T06:28:06.492 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.485+0000 7f0d81adc700 1 -- 172.21.15.114:0/2165121602 wait complete. 2023-11-24T06:28:06.497 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.486+0000 7f0d81adc700 1 Processor -- start 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.486+0000 7f0d81adc700 1 -- start start 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c0af4d0 0x7f0d7c15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15e080 0x7f0d7c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c158950 con 0x7f0d7c06c000 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c157cd0 con 0x7f0d7c15e080 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d81adc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c157fb0 con 0x7f0d7c0af4d0 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42678/0 (socket says 172.21.15.114:42678) 2023-11-24T06:28:06.498 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 -- 172.21.15.114:0/246065724 learned_addr learned my addr 172.21.15.114:0/246065724 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7bfff700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15e080 0x7f0d7c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7affd700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c0af4d0 0x7f0d7c15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 -- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c0af4d0 msgr2=0x7f0d7c15ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c0af4d0 0x7f0d7c15ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 -- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15e080 msgr2=0x7f0d7c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15e080 0x7f0d7c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.487+0000 7f0d7b7fe700 1 -- 172.21.15.114:0/246065724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009b30 con 0x7f0d7c06c000 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d7b7fe700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c7f0 tx=0x7f0d6c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c011800 con 0x7f0d7c06c000 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c011960 con 0x7f0d7c06c000 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c158290 con 0x7f0d7c06c000 2023-11-24T06:28:06.499 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c011cf0 con 0x7f0d7c06c000 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.488+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c156a00 con 0x7f0d7c06c000 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.489+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c0753f0 con 0x7f0d7c06c000 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.489+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0d6c018360 con 0x7f0d7c06c000 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.490+0000 7f0d63fff700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 0x7f0d6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.490+0000 7f0d7affd700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 0x7f0d6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.490+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d6c089860 con 0x7f0d7c06c000 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.490+0000 7f0d7affd700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 0x7f0d6806d540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c15a250 tx=0x7f0d70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.500 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.494+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d6c054610 con 0x7f0d7c06c000 2023-11-24T06:28:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.726+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7c158a90 con 0x7f0d7c06c000 2023-11-24T06:28:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.726+0000 7f0d63fff700 1 -- 172.21.15.114:0/246065724 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f0d7c158a90 con 0x7f0d7c06c000 2023-11-24T06:28:06.729 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:06.730 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:27:44.273957+0000","modified":"2023-11-24T06:27:47.590785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6216},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6216":{"gid":6216,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6837/3988687339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":3988687339},{"type":"v1","addr":"172.21.15.114:6837","nonce":3988687339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 msgr2=0x7f0d6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 0x7f0d6806d540 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c15a250 tx=0x7f0d70008040 comp rx=0 tx=0).stop 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 msgr2=0x7f0d7c157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c7f0 tx=0x7f0d6c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 shutdown_connections 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0d6806b080 0x7f0d6806d540 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c15e080 0x7f0d7c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d7c06c000 0x7f0d7c157250 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 --2- 172.21.15.114:0/246065724 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c0af4d0 0x7f0d7c15ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 >> 172.21.15.114:0/246065724 conn(0x7f0d7c071b90 msgr2=0x7f0d7c0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 shutdown_connections 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.728+0000 7f0d81adc700 1 -- 172.21.15.114:0/246065724 wait complete. 2023-11-24T06:28:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2023-11-24T06:28:06.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:06.863 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.861+0000 7fc7cb9bc700 1 Processor -- start 2023-11-24T06:28:06.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.861+0000 7fc7cb9bc700 1 -- start start 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 0x7fc7c40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7c40adf00 0x7fc7c406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 0x7fc7c406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c4144a80 con 0x7fc7c40adf00 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c406c9c0 con 0x7fc7c41446a0 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7cb9bc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40af950 con 0x7fc7c406c540 2023-11-24T06:28:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7c40adf00 0x7fc7c406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7c40adf00 0x7fc7c406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42686/0 (socket says 172.21.15.114:42686) 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 -- 172.21.15.114:0/2350625930 learned_addr learned my addr 172.21.15.114:0/2350625930 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 -- 172.21.15.114:0/2350625930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 msgr2=0x7fc7c406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c9f59700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 0x7fc7c406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c9758700 1 --2- 172.21.15.114:0/2350625930 >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 0x7fc7c40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 0x7fc7c406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 -- 172.21.15.114:0/2350625930 >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 msgr2=0x7fc7c40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/2350625930 >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 0x7fc7c40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c8f57700 1 -- 172.21.15.114:0/2350625930 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40afc30 con 0x7fc7c40adf00 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c9758700 1 --2- 172.21.15.114:0/2350625930 >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 0x7fc7c40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.862+0000 7fc7c9f59700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 0x7fc7c406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c40adf00 0x7fc7c406c000 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40729a0 tx=0x7fc7c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb0fe1c9133c386 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7bbfff700 1 -- 172.21.15.114:0/2350625930 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000b870 con 0x7fc7c40adf00 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7bbfff700 1 -- 172.21.15.114:0/2350625930 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000b9d0 con 0x7fc7c40adf00 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7bbfff700 1 -- 172.21.15.114:0/2350625930 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000bd60 con 0x7fc7c40adf00 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/2350625930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c40adf00 msgr2=0x7fc7c406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c40adf00 0x7fc7c406c000 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40729a0 tx=0x7fc7c0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/2350625930 shutdown_connections 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c406c540 0x7fc7c406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/2350625930 >> v2:172.21.15.172:3300/0 conn(0x7fc7c41446a0 0x7fc7c40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/2350625930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c40adf00 0x7fc7c406c000 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/2350625930 >> 172.21.15.114:0/2350625930 conn(0x7fc7c4071b90 msgr2=0x7fc7c4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/2350625930 shutdown_connections 2023-11-24T06:28:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.863+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/2350625930 wait complete. 2023-11-24T06:28:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 Processor -- start 2023-11-24T06:28:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 -- start start 2023-11-24T06:28:06.872 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 0x7fc7c415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c41446a0 0x7fc7c4137c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c41f2ac0 con 0x7fc7c406c540 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c4138900 con 0x7fc7c40adf00 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7cb9bc700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c4138be0 con 0x7fc7c41446a0 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7c9758700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7c9758700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42692/0 (socket says 172.21.15.114:42692) 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.865+0000 7fc7c9758700 1 -- 172.21.15.114:0/3621600127 learned_addr learned my addr 172.21.15.114:0/3621600127 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9f59700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c41446a0 0x7fc7c4137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 0x7fc7c415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 -- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c41446a0 msgr2=0x7fc7c4137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c41446a0 0x7fc7c4137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 -- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 msgr2=0x7fc7c415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 0x7fc7c415da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 -- 172.21.15.114:0/3621600127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009550 con 0x7fc7c406c540 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 0x7fc7c415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7c9758700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c7f0 tx=0x7fc7b400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4011800 con 0x7fc7c406c540 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4011960 con 0x7fc7c406c540 2023-11-24T06:28:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c4138ec0 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4011cf0 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.866+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c40d0890 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.867+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c40753f0 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.867+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc7b4018360 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.868+0000 7fc7b9ffb700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 0x7fc7a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.868+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7b4089990 con 0x7fc7c406c540 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.868+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 0x7fc7a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.875 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.868+0000 7fc7c8f57700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 0x7fc7a806d540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40729a0 tx=0x7fc7c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.876 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:06.872+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7b40546c0 con 0x7fc7c406c540 2023-11-24T06:28:07.068 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.065+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c4067f60 con 0x7fc7c406c540 2023-11-24T06:28:07.069 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.066+0000 7fc7b9ffb700 1 -- 172.21.15.114:0/3621600127 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fc7b4057ce0 con 0x7fc7c406c540 2023-11-24T06:28:07.069 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:07.069 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:27:46.179953+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:27:41.963773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:27:43.243874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.068+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 msgr2=0x7fc7a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.068+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 0x7fc7a806d540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40729a0 tx=0x7fc7c0008030 comp rx=0 tx=0).stop 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.068+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 msgr2=0x7fc7c415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.068+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c7f0 tx=0x7fc7b400cb00 comp rx=0 tx=0).stop 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 shutdown_connections 2023-11-24T06:28:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc7a806b080 0x7fc7a806d540 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7c41446a0 0x7fc7c4137c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7c406c540 0x7fc7c415d550 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 --2- 172.21.15.114:0/3621600127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7c40adf00 0x7fc7c415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 >> 172.21.15.114:0/3621600127 conn(0x7fc7c4071b90 msgr2=0x7fc7c406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 shutdown_connections 2023-11-24T06:28:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.069+0000 7fc7cb9bc700 1 -- 172.21.15.114:0/3621600127 wait complete. 2023-11-24T06:28:07.083 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:28:07.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.204+0000 7f7feb2a3700 1 Processor -- start 2023-11-24T06:28:07.206 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.204+0000 7f7feb2a3700 1 -- start start 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7feb2a3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe415bdd0 0x7f7fe41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7feb2a3700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7fe40b3370 0x7f7fe40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7feb2a3700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe40b2030 con 0x7f7fe40d1c70 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7feb2a3700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe40b2f50 con 0x7f7fe40b3370 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7feb2a3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40b22d0 con 0x7f7fe415bdd0 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7fe40b3370 0x7f7fe40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.209 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe9840700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7fe40d1c70 0x7f7fe40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58412/0 (socket says 172.21.15.114:58412) 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe9840700 1 -- 172.21.15.114:0/453265119 learned_addr learned my addr 172.21.15.114:0/453265119 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fdbfff700 1 -- 172.21.15.114:0/453265119 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593652548 0 0) 0x7f7fe40b2030 con 0x7f7fe40d1c70 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe903f700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe415bdd0 0x7f7fe41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fdbfff700 1 -- 172.21.15.114:0/453265119 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd00035b0 con 0x7f7fe40d1c70 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 -- 172.21.15.114:0/453265119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe415bdd0 msgr2=0x7f7fe41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe415bdd0 0x7f7fe41604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 -- 172.21.15.114:0/453265119 >> v1:172.21.15.114:6789/0 conn(0x7f7fe40d1c70 legacy=0x7f7fe40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 -- 172.21.15.114:0/453265119 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe40b25b0 con 0x7f7fe40b3370 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.205+0000 7f7fe883e700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe40b1af0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009840 tx=0x7f7fd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2381dbb7b78d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7fdbfff700 1 -- 172.21.15.114:0/453265119 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011830 con 0x7f7fe40b3370 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7fdbfff700 1 -- 172.21.15.114:0/453265119 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4011990 con 0x7f7fe40b3370 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7fdbfff700 1 -- 172.21.15.114:0/453265119 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011d20 con 0x7f7fe40b3370 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 -- 172.21.15.114:0/453265119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 msgr2=0x7f7fe40b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe40b1af0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009840 tx=0x7f7fd4009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 -- 172.21.15.114:0/453265119 shutdown_connections 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe415bdd0 0x7f7fe41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/453265119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe40b1af0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 -- 172.21.15.114:0/453265119 >> 172.21.15.114:0/453265119 conn(0x7f7fe40731a0 msgr2=0x7f7fe409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 -- 172.21.15.114:0/453265119 shutdown_connections 2023-11-24T06:28:07.211 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.206+0000 7f7feb2a3700 1 -- 172.21.15.114:0/453265119 wait complete. 2023-11-24T06:28:07.215 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.207+0000 7f7feb2a3700 1 Processor -- start 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.207+0000 7f7feb2a3700 1 -- start start 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe40d1c70 0x7f7fe4150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fe415bdd0 0x7f7fe40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4150c60 con 0x7f7fe40b3370 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40ce3a0 con 0x7f7fe415bdd0 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7feb2a3700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40ce680 con 0x7f7fe40d1c70 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7fe903f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7fe903f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42706/0 (socket says 172.21.15.114:42706) 2023-11-24T06:28:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7fe903f700 1 -- 172.21.15.114:0/3802236692 learned_addr learned my addr 172.21.15.114:0/3802236692 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.208+0000 7f7fe883e700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe40d1c70 0x7f7fe4150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 -- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe40d1c70 msgr2=0x7f7fe4150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe40d1c70 0x7f7fe4150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 -- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fe415bdd0 msgr2=0x7f7fe40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fe415bdd0 0x7f7fe40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 -- 172.21.15.114:0/3802236692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40094a0 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fe903f700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c7f0 tx=0x7f7fe000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0011800 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe0011960 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe40c9e00 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0011cf0 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.209+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe40ca3c0 con 0x7f7fe40b3370 2023-11-24T06:28:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.210+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe4067f60 con 0x7f7fe40b3370 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.210+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f7fe0018360 con 0x7f7fe40b3370 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.211+0000 7f7fd9ffb700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 0x7f7fc806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.211+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fe0089890 con 0x7f7fe40b3370 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.211+0000 7f7fe883e700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 0x7f7fc806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.211+0000 7f7fe883e700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 0x7f7fc806d540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fe415f6b0 tx=0x7f7fd4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.215+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7fe0054640 con 0x7f7fe40b3370 2023-11-24T06:28:07.451 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.449+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7fe40ca5e0 con 0x7f7fe40b3370 2023-11-24T06:28:07.452 INFO:tasks.ceph.mon.a.smithi114.stderr:2023-11-24T06:28:07.450+0000 7f8fd8393700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:28:07.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.451+0000 7f7fd9ffb700 1 -- 172.21.15.114:0/3802236692 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7fe001f090 con 0x7f7fe40b3370 2023-11-24T06:28:07.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 msgr2=0x7f7fc806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 0x7f7fc806d540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fe415f6b0 tx=0x7f7fd4008030 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 msgr2=0x7f7fe406cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c7f0 tx=0x7f7fe000cbb0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 shutdown_connections 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f7fc806b080 0x7f7fc806d540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fe415bdd0 0x7f7fe40cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe40b3370 0x7f7fe406cc80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 --2- 172.21.15.114:0/3802236692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fe40d1c70 0x7f7fe4150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 >> 172.21.15.114:0/3802236692 conn(0x7f7fe40731a0 msgr2=0x7f7fe40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 shutdown_connections 2023-11-24T06:28:07.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.454+0000 7f7feb2a3700 1 -- 172.21.15.114:0/3802236692 wait complete. 2023-11-24T06:28:07.458 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:28:07.469 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:28:07.487 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2023-11-24T06:28:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.592+0000 7fe72c9a0700 1 Processor -- start 2023-11-24T06:28:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.592+0000 7fe72c9a0700 1 -- start start 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72c9a0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe72815b370 0x7fe72815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72c9a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72c9a0700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280ae5e0 con 0x7fe7281587f0 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72c9a0700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7280b4630 con 0x7fe72815b370 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72c9a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72815be30 con 0x7fe7280b3c60 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72659c700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe7281587f0 0x7fe72806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58434/0 (socket says 172.21.15.114:58434) 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72659c700 1 -- 172.21.15.114:0/1868893195 learned_addr learned my addr 172.21.15.114:0/1868893195 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 --2- 172.21.15.114:0/1868893195 >> v2:172.21.15.114:3300/0 conn(0x7fe72815b370 0x7fe72815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe726d9d700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72559a700 1 -- 172.21.15.114:0/1868893195 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918601697 0 0) 0x7fe7280ae5e0 con 0x7fe7281587f0 2023-11-24T06:28:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72559a700 1 -- 172.21.15.114:0/1868893195 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7100035b0 con 0x7fe7281587f0 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 -- 172.21.15.114:0/1868893195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 msgr2=0x7fe7280b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 -- 172.21.15.114:0/1868893195 >> v1:172.21.15.114:6789/0 conn(0x7fe7281587f0 legacy=0x7fe72806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 -- 172.21.15.114:0/1868893195 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72815c110 con 0x7fe72815b370 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe725d9b700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe72815b800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe718009870 tx=0x7fe718009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2862a47226d416c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72559a700 1 -- 172.21.15.114:0/1868893195 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718011830 con 0x7fe72815b370 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72559a700 1 -- 172.21.15.114:0/1868893195 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718011990 con 0x7fe72815b370 2023-11-24T06:28:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.593+0000 7fe72559a700 1 -- 172.21.15.114:0/1868893195 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718011d20 con 0x7fe72815b370 2023-11-24T06:28:07.600 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 -- 172.21.15.114:0/1868893195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 msgr2=0x7fe72815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe72815b800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe718009870 tx=0x7fe718009b80 comp rx=0 tx=0).stop 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 -- 172.21.15.114:0/1868893195 shutdown_connections 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/1868893195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe72815b800 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 -- 172.21.15.114:0/1868893195 >> 172.21.15.114:0/1868893195 conn(0x7fe7280d0bf0 msgr2=0x7fe7280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 -- 172.21.15.114:0/1868893195 shutdown_connections 2023-11-24T06:28:07.601 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.594+0000 7fe72c9a0700 1 -- 172.21.15.114:0/1868893195 wait complete. 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.595+0000 7fe72c9a0700 1 Processor -- start 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.595+0000 7fe72c9a0700 1 -- start start 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 0x7fe72814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280b1f60 con 0x7fe72815b370 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280b0c40 con 0x7fe7281587f0 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72c9a0700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72814d7c0 con 0x7fe7280b3c60 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe726d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe726d9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42736/0 (socket says 172.21.15.114:42736) 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe726d9d700 1 -- 172.21.15.114:0/954387953 learned_addr learned my addr 172.21.15.114:0/954387953 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe72659c700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.596+0000 7fe725d9b700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 0x7fe72814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 -- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 msgr2=0x7fe7280b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 -- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 msgr2=0x7fe72814d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 0x7fe72814d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 -- 172.21.15.114:0/954387953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180094a0 con 0x7fe72815b370 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe72659c700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe725d9b700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 0x7fe72814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe726d9d700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe720007e70 tx=0x7fe72000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72000ee50 con 0x7fe72815b370 2023-11-24T06:28:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe720014420 con 0x7fe72815b370 2023-11-24T06:28:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7280b0ec0 con 0x7fe72815b370 2023-11-24T06:28:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200147d0 con 0x7fe72815b370 2023-11-24T06:28:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.597+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7280b1760 con 0x7fe72815b370 2023-11-24T06:28:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.598+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe728068fb0 con 0x7fe72815b370 2023-11-24T06:28:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.598+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe720014930 con 0x7fe72815b370 2023-11-24T06:28:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.599+0000 7fe716ffd700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 0x7fe70806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.599+0000 7fe72659c700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 0x7fe70806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.599+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe72008a120 con 0x7fe72815b370 2023-11-24T06:28:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.599+0000 7fe72659c700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 0x7fe70806d540 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0011e0 tx=0x7fe71c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.606 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.603+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe720054ef0 con 0x7fe72815b370 2023-11-24T06:28:07.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:07.839+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe7280d0650 con 0x7fe72815b370 2023-11-24T06:28:08.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.455+0000 7fe716ffd700 1 -- 172.21.15.114:0/954387953 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe720058510 con 0x7fe72815b370 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 msgr2=0x7fe70806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 0x7fe70806d540 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0011e0 tx=0x7fe71c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 msgr2=0x7fe728149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe720007e70 tx=0x7fe72000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 shutdown_connections 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe70806b080 0x7fe70806d540 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe72815b370 0x7fe728149920 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7280b3c60 0x7fe7280b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 --2- 172.21.15.114:0/954387953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7281587f0 0x7fe72814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 >> 172.21.15.114:0/954387953 conn(0x7fe7280d0bf0 msgr2=0x7fe72806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:08.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 shutdown_connections 2023-11-24T06:28:08.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.458+0000 7fe72c9a0700 1 -- 172.21.15.114:0/954387953 wait complete. 2023-11-24T06:28:08.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:28:08.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.638+0000 7f9440217700 1 Processor -- start 2023-11-24T06:28:08.640 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.638+0000 7f9440217700 1 -- start start 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f9440217700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f943806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f9440217700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943806b200 con 0x7f943806e830 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f9440217700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943806b4e0 con 0x7f9438144600 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f9440217700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943806aff0 con 0x7f943806f480 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943e7b4700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f943806e830 0x7f94380cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58454/0 (socket says 172.21.15.114:58454) 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943e7b4700 1 -- 172.21.15.114:0/65582159 learned_addr learned my addr 172.21.15.114:0/65582159 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943d7b2700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f943806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483226640 0 0) 0x7f943806b200 con 0x7f943806e830 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94240035b0 con 0x7f943806e830 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587644383 0 0) 0x7f943806b4e0 con 0x7f9438144600 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f943806b200 con 0x7f9438144600 2023-11-24T06:28:08.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594958 0 0) 0x7f94240035b0 con 0x7f943806e830 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 msgr2=0x7f943806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f943806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 >> v1:172.21.15.172:6789/0 conn(0x7f9438144600 legacy=0x7f94380b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943806b7c0 con 0x7f943806e830 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f94300019f0 con 0x7f943806e830 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.639+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 >> v1:172.21.15.114:6789/0 conn(0x7f943806e830 legacy=0x7f94380cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f9424005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943806aff0 con 0x7f9424004850 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942400dd00 con 0x7f9424003bc0 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942400dec0 con 0x7f943806f480 2023-11-24T06:28:08.645 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943dfb3700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943e7b4700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f9424005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 -- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 msgr2=0x7f9424005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f9424005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 -- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424003bc0 msgr2=0x7f9424003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 -- 172.21.15.114:0/65582159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240035b0 con 0x7f9424004850 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943e7b4700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f943806f480 0x7f9424005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943d7b2700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9424004850 0x7f9424004cd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9428009b60 tx=0x7f9424005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71140c4ff24f50a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800e6c0 con 0x7f9424004850 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800e820 con 0x7f9424004850 2023-11-24T06:28:08.646 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.640+0000 7f943cfb1700 1 -- 172.21.15.114:0/65582159 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800eb90 con 0x7f9424004850 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 -- 172.21.15.114:0/65582159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9424004850 msgr2=0x7f9424004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 --2- 172.21.15.114:0/65582159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9424004850 0x7f9424004cd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9428009b60 tx=0x7f9424005880 comp rx=0 tx=0).stop 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f943e7b4700 1 -- 172.21.15.114:0/65582159 reap_dead start 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 -- 172.21.15.114:0/65582159 shutdown_connections 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 -- 172.21.15.114:0/65582159 >> 172.21.15.114:0/65582159 conn(0x7f94380d0b40 msgr2=0x7f94380d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 -- 172.21.15.114:0/65582159 shutdown_connections 2023-11-24T06:28:08.647 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.641+0000 7f9440217700 1 -- 172.21.15.114:0/65582159 wait complete. 2023-11-24T06:28:08.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.642+0000 7f9440217700 1 Processor -- start 2023-11-24T06:28:08.649 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.642+0000 7f9440217700 1 -- start start 2023-11-24T06:28:08.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 0x7f943815d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94380ad870 0x7f94380b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.650 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943815dd40 con 0x7f94381391d0 2023-11-24T06:28:08.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438138170 con 0x7f9424004850 2023-11-24T06:28:08.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380ade30 con 0x7f94380ad870 2023-11-24T06:28:08.651 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42758/0 (socket says 172.21.15.114:42758) 2023-11-24T06:28:08.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 -- 172.21.15.114:0/3919990196 learned_addr learned my addr 172.21.15.114:0/3919990196 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:08.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943d7b2700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94380ad870 0x7f94380b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.652 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943dfb3700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 0x7f943815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 -- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94380ad870 msgr2=0x7f94380b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94380ad870 0x7f94380b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 -- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 msgr2=0x7f943815d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 0x7f943815d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 -- 172.21.15.114:0/3919990196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380ae110 con 0x7f94381391d0 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943dfb3700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 0x7f943815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:08.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f943e7b4700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9438072fa0 tx=0x7f9430007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943000ec60 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943000edc0 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380ae3f0 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.643+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430013660 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.644+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380b1800 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.644+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438067f60 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.645+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f94300137c0 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.645+0000 7f942e7fc700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 0x7f941c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.645+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9430089fc0 con 0x7f94381391d0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.645+0000 7f943dfb3700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 0x7f941c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.656 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.646+0000 7f943dfb3700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 0x7f941c06d540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9434000c00 tx=0x7f9434009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.650+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9430054cf0 con 0x7f94381391d0 2023-11-24T06:28:08.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:08.840+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f943806e100 con 0x7f94381391d0 2023-11-24T06:28:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.457+0000 7f942e7fc700 1 -- 172.21.15.114:0/3919990196 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9430058310 con 0x7f94381391d0 2023-11-24T06:28:09.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 msgr2=0x7f941c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 0x7f941c06d540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9434000c00 tx=0x7f9434009040 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 msgr2=0x7f9438138600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9438072fa0 tx=0x7f9430007c90 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 shutdown_connections 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f941c06b080 0x7f941c06d540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94381391d0 0x7f9438138600 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9424004850 0x7f943815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 --2- 172.21.15.114:0/3919990196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94380ad870 0x7f94380b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 >> 172.21.15.114:0/3919990196 conn(0x7f94380d0b40 msgr2=0x7f94380cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 shutdown_connections 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.460+0000 7f9440217700 1 -- 172.21.15.114:0/3919990196 wait complete. 2023-11-24T06:28:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:28:09.477 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:28:09.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.607+0000 7f0f2b2f4700 1 Processor -- start 2023-11-24T06:28:09.610 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.608+0000 7f0f2b2f4700 1 -- start start 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2b2f4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240746a0 0x7f0f2406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2b2f4700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f240b59c0 0x7f0f240b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2b2f4700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240b5e00 con 0x7f0f240b01e0 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2b2f4700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f241470a0 con 0x7f0f240b59c0 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2b2f4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240b05c0 con 0x7f0f240746a0 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f240b59c0 0x7f0f240b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29090700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0f240b01e0 0x7f0f24073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58470/0 (socket says 172.21.15.114:58470) 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29090700 1 -- 172.21.15.114:0/1661434113 learned_addr learned my addr 172.21.15.114:0/1661434113 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f2888f700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240746a0 0x7f0f2406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 -- 172.21.15.114:0/1661434113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240746a0 msgr2=0x7f0f2406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240746a0 0x7f0f2406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.613 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 -- 172.21.15.114:0/1661434113 >> v1:172.21.15.114:6789/0 conn(0x7f0f240b01e0 legacy=0x7f0f24073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 -- 172.21.15.114:0/1661434113 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f240b4250 con 0x7f0f240b59c0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f1bfff700 1 -- 172.21.15.114:0/1661434113 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986219974 0 0) 0x7f0f240b5e00 con 0x7f0f240b01e0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f29891700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240b3b50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009d90 tx=0x7f0f1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9b468345f92076 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f1bfff700 1 -- 172.21.15.114:0/1661434113 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00e070 con 0x7f0f240b59c0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f1bfff700 1 -- 172.21.15.114:0/1661434113 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00bbd0 con 0x7f0f240b59c0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.609+0000 7f0f1bfff700 1 -- 172.21.15.114:0/1661434113 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c012430 con 0x7f0f240b59c0 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/1661434113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 msgr2=0x7f0f240b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240b3b50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009d90 tx=0x7f0f1c00b330 comp rx=0 tx=0).stop 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/1661434113 shutdown_connections 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240b3b50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/1661434113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240746a0 0x7f0f2406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/1661434113 >> 172.21.15.114:0/1661434113 conn(0x7f0f240d0b40 msgr2=0x7f0f240d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:09.614 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/1661434113 shutdown_connections 2023-11-24T06:28:09.615 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.610+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/1661434113 wait complete. 2023-11-24T06:28:09.617 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.611+0000 7f0f2b2f4700 1 Processor -- start 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.611+0000 7f0f2b2f4700 1 -- start start 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 0x7f0f24137390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240b01e0 0x7f0f240cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240c97d0 con 0x7f0f240b59c0 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240cbab0 con 0x7f0f240746a0 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240c8bc0 con 0x7f0f240b01e0 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42772/0 (socket says 172.21.15.114:42772) 2023-11-24T06:28:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 -- 172.21.15.114:0/3736975741 learned_addr learned my addr 172.21.15.114:0/3736975741 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29090700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 0x7f0f24137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2888f700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240b01e0 0x7f0f240cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 -- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240b01e0 msgr2=0x7f0f240cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240b01e0 0x7f0f240cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 -- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 msgr2=0x7f0f24137390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 0x7f0f24137390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 -- 172.21.15.114:0/3736975741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c009a70 con 0x7f0f240b59c0 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29090700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 0x7f0f24137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f29891700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0f24072d10 tx=0x7f0f1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c012430 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c020070 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f240c8ea0 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.612+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c004280 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.613+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f240c9460 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.614+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f240c9980 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.614+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0f1c0043e0 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.614+0000 7f0f19ffb700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 0x7f0f0806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.615+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0f1c022070 con 0x7f0f240b59c0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.615+0000 7f0f29090700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 0x7f0f0806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.615+0000 7f0f29090700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 0x7f0f0806d540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f14000f60 tx=0x7f0f14007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.621 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.619+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f240c9980 con 0x7f0f240b59c0 2023-11-24T06:28:09.815 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:09.813+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f240b45d0 con 0x7f0f240b59c0 2023-11-24T06:28:10.467 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.465+0000 7f0f19ffb700 1 -- 172.21.15.114:0/3736975741 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0f1c0568f0 con 0x7f0f240b59c0 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 msgr2=0x7f0f0806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 0x7f0f0806d540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f14000f60 tx=0x7f0f14007ca0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 msgr2=0x7f0f240c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0f24072d10 tx=0x7f0f1c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 shutdown_connections 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f0f0806b080 0x7f0f0806d540 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f240b59c0 0x7f0f240c9c60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f240746a0 0x7f0f24137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 --2- 172.21.15.114:0/3736975741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f240b01e0 0x7f0f240cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.469+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 >> 172.21.15.114:0/3736975741 conn(0x7f0f240d0b40 msgr2=0x7f0f240adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.470+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 shutdown_connections 2023-11-24T06:28:10.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.470+0000 7f0f2b2f4700 1 -- 172.21.15.114:0/3736975741 wait complete. 2023-11-24T06:28:10.473 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2023-11-24T06:28:10.488 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:28:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.613+0000 7fc9ae9ab700 1 Processor -- start 2023-11-24T06:28:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.613+0000 7fc9ae9ab700 1 -- start start 2023-11-24T06:28:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9ae9ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc9a80ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9ae9ab700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80b2f20 con 0x7fc9a806ecf0 2023-11-24T06:28:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9ae9ab700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80ce690 con 0x7fc9a80cc540 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9ae9ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a815aa00 con 0x7fc9a815a620 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9acf48700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc9a806ecf0 0x7fc9a80b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58486/0 (socket says 172.21.15.114:58486) 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9acf48700 1 -- 172.21.15.114:0/3779370818 learned_addr learned my addr 172.21.15.114:0/3779370818 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697178856 0 0) 0x7fc9a80b2f20 con 0x7fc9a806ecf0 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.614+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9940035b0 con 0x7fc9a806ecf0 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc9a80ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645760653 0 0) 0x7fc9940035b0 con 0x7fc9a806ecf0 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 msgr2=0x7fc9a80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc9a80ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 >> v1:172.21.15.172:6789/0 conn(0x7fc9a80cc540 legacy=0x7fc9a806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80ce960 con 0x7fc9a806ecf0 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fc9a00019f0 con 0x7fc9a806ecf0 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 >> v1:172.21.15.114:6789/0 conn(0x7fc9a806ecf0 legacy=0x7fc9a80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc994003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc994004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc994004d90 0x7fc994005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a815aa00 con 0x7fc994003a30 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400bba0 con 0x7fc994004d90 2023-11-24T06:28:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400df40 con 0x7fc9a815a620 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc994003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9acf48700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc994004d90 0x7fc994005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc99ffff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc994004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 -- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 msgr2=0x7fc994004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc994004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 -- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc994004d90 msgr2=0x7fc994005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc994004d90 0x7fc994005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 -- 172.21.15.114:0/3779370818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940035b0 con 0x7fc994003a30 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc99ffff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a815a620 0x7fc994004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc994003e50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc9980098c0 tx=0x7fc994005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee561b56f4e532c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800c070 con 0x7fc994003a30 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc998013610 con 0x7fc994003a30 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.615+0000 7fc9a77fe700 1 -- 172.21.15.114:0/3779370818 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9980139a0 con 0x7fc994003a30 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/3779370818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 msgr2=0x7fc994003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/3779370818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc994003e50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc9980098c0 tx=0x7fc994005880 comp rx=0 tx=0).stop 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9acf48700 1 -- 172.21.15.114:0/3779370818 reap_dead start 2023-11-24T06:28:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/3779370818 shutdown_connections 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/3779370818 >> 172.21.15.114:0/3779370818 conn(0x7fc9a80b2050 msgr2=0x7fc9a80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/3779370818 shutdown_connections 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.616+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/3779370818 wait complete. 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.617+0000 7fc9ae9ab700 1 Processor -- start 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.617+0000 7fc9ae9ab700 1 -- start start 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.636 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a814a300 0x7fc9a815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.637 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a8149980 0x7fc9a814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a815bbc0 con 0x7fc994003a30 2023-11-24T06:28:10.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a814e600 con 0x7fc9a814a300 2023-11-24T06:28:10.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9ae9ab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a814e8e0 con 0x7fc9a8149980 2023-11-24T06:28:10.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.642 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42788/0 (socket says 172.21.15.114:42788) 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 -- 172.21.15.114:0/1303468885 learned_addr learned my addr 172.21.15.114:0/1303468885 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 -- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a8149980 msgr2=0x7fc9a814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9acf48700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a8149980 0x7fc9a814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc99ffff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a814a300 0x7fc9a815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a8149980 0x7fc9a814f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 -- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a814a300 msgr2=0x7fc9a815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a814a300 0x7fc9a815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.618+0000 7fc9a7fff700 1 -- 172.21.15.114:0/1303468885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a814ebc0 con 0x7fc994003a30 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9a7fff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc998009eb0 tx=0x7fc9980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998013c50 con 0x7fc994003a30 2023-11-24T06:28:10.643 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc998013dd0 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a814eea0 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9980038e0 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.619+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a814a780 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.620+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a8068d90 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.620+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc998003a40 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.620+0000 7fc9a57fa700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 0x7fc98806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.621+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc99808aaa0 con 0x7fc994003a30 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.621+0000 7fc99ffff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 0x7fc98806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.621+0000 7fc99ffff700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 0x7fc98806d570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9a815ac80 tx=0x7fc990009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.644 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.625+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc998055b50 con 0x7fc994003a30 2023-11-24T06:28:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.815+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc9a814f3c0 con 0x7fc994003a30 2023-11-24T06:28:10.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.818+0000 7fc9a57fa700 1 -- 172.21.15.114:0/1303468885 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9a814f3c0 con 0x7fc994003a30 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 msgr2=0x7fc98806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 0x7fc98806d570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9a815ac80 tx=0x7fc990009040 comp rx=0 tx=0).stop 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 msgr2=0x7fc9a814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc998009eb0 tx=0x7fc9980047c0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 shutdown_connections 2023-11-24T06:28:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc98806b0b0 0x7fc98806d570 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9a815ac80 tx=0x7fc990009040 comp rx=0 tx=0).stop 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a8149980 0x7fc9a814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc994003a30 0x7fc9a814df70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 --2- 172.21.15.114:0/1303468885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a814a300 0x7fc9a815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 >> 172.21.15.114:0/1303468885 conn(0x7fc9a80b2050 msgr2=0x7fc9a806e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 shutdown_connections 2023-11-24T06:28:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.820+0000 7fc9ae9ab700 1 -- 172.21.15.114:0/1303468885 wait complete. 2023-11-24T06:28:10.835 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-24T06:28:10.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.953+0000 7f02c42c5700 1 Processor -- start 2023-11-24T06:28:10.956 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.953+0000 7f02c42c5700 1 -- start start 2023-11-24T06:28:10.963 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c42c5700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 0x7f02bc0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c42c5700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f02bc06c710 0x7f02bc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c42c5700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0af950 con 0x7f02bc0adf00 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c42c5700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc144a80 con 0x7f02bc06c710 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c42c5700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc0afc30 con 0x7f02bc1446a0 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c2862700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f02bc06c710 0x7f02bc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c2862700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f02bc06c710 0x7f02bc0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42800/0 (socket says 172.21.15.114:42800) 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.954+0000 7f02c2862700 1 -- 172.21.15.114:0/1397387749 learned_addr learned my addr 172.21.15.114:0/1397387749 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2862700 1 -- 172.21.15.114:0/1397387749 >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 msgr2=0x7f02bc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2061700 1 --2- 172.21.15.114:0/1397387749 >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 0x7f02bc0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.964 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2862700 1 --2- 172.21.15.114:0/1397387749 >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 0x7f02bc0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2862700 1 -- 172.21.15.114:0/1397387749 >> v1:172.21.15.172:6789/0 conn(0x7f02bc0adf00 legacy=0x7f02bc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2862700 1 -- 172.21.15.114:0/1397387749 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0aff10 con 0x7f02bc06c710 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2061700 1 --2- 172.21.15.114:0/1397387749 >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 0x7f02bc0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c2862700 1 --2- 172.21.15.114:0/1397387749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc0b3d40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0729a0 tx=0x7f02b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186fb0b737b75643 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c105f700 1 -- 172.21.15.114:0/1397387749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b400e070 con 0x7f02bc06c710 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c105f700 1 -- 172.21.15.114:0/1397387749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4003a20 con 0x7f02bc06c710 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c105f700 1 -- 172.21.15.114:0/1397387749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4003dd0 con 0x7f02bc06c710 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c42c5700 1 -- 172.21.15.114:0/1397387749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 msgr2=0x7f02bc0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.955+0000 7f02c42c5700 1 --2- 172.21.15.114:0/1397387749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc0b3d40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0729a0 tx=0x7f02b400b330 comp rx=0 tx=0).stop 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 -- 172.21.15.114:0/1397387749 shutdown_connections 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 --2- 172.21.15.114:0/1397387749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc0b3d40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 --2- 172.21.15.114:0/1397387749 >> v2:172.21.15.172:3300/0 conn(0x7f02bc1446a0 0x7f02bc0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.965 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 -- 172.21.15.114:0/1397387749 >> 172.21.15.114:0/1397387749 conn(0x7f02bc071b90 msgr2=0x7f02bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 -- 172.21.15.114:0/1397387749 shutdown_connections 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.956+0000 7f02c42c5700 1 -- 172.21.15.114:0/1397387749 wait complete. 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 Processor -- start 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 -- start start 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02bc0adf00 0x7f02bc0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02bc1446a0 0x7f02bc159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0cfcc0 con 0x7f02bc06c710 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0cffa0 con 0x7f02bc0adf00 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.957+0000 7f02c42c5700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0d0280 con 0x7f02bc1446a0 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.966 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42802/0 (socket says 172.21.15.114:42802) 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 -- 172.21.15.114:0/3872614343 learned_addr learned my addr 172.21.15.114:0/3872614343 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2862700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02bc1446a0 0x7f02bc159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c1860700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02bc0adf00 0x7f02bc0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 -- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02bc1446a0 msgr2=0x7f02bc159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02bc1446a0 0x7f02bc159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 -- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02bc0adf00 msgr2=0x7f02bc0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02bc0adf00 0x7f02bc0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 -- 172.21.15.114:0/3872614343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009b30 con 0x7f02bc06c710 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c2061700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c7f0 tx=0x7f02ac00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac011800 con 0x7f02bc06c710 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac011960 con 0x7f02bc06c710 2023-11-24T06:28:10.967 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac011cf0 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc0cdb40 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.958+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc143280 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.959+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc0753f0 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.960+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f02ac018360 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.960+0000 7f02b2ffd700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.960+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02ac0894d0 con 0x7f02bc06c710 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.960+0000 7f02c1860700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.968 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.960+0000 7f02c1860700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 0x7f02a006d540 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0cd3f0 tx=0x7f02b8007ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.969 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:10.964+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02ac054580 con 0x7f02bc06c710 2023-11-24T06:28:11.242 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.240+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f02bc0cfe00 con 0x7f02bc06c710 2023-11-24T06:28:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.466+0000 7f02b2ffd700 1 -- 172.21.15.114:0/3872614343 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v218) v1 ==== 161+0+0 (secure 0 0 0) 0x7f02bc0cfe00 con 0x7f02bc06c710 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 msgr2=0x7f02a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 0x7f02a006d540 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0cd3f0 tx=0x7f02b8007ee0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 msgr2=0x7f02bc1675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c7f0 tx=0x7f02ac00cb00 comp rx=0 tx=0).stop 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 shutdown_connections 2023-11-24T06:28:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02bc1446a0 0x7f02bc159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f02bc06c710 0x7f02bc1675a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 --2- 172.21.15.114:0/3872614343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02bc0adf00 0x7f02bc0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.469+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 >> 172.21.15.114:0/3872614343 conn(0x7f02bc071b90 msgr2=0x7f02bc0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.470+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 shutdown_connections 2023-11-24T06:28:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:11.470+0000 7f02c42c5700 1 -- 172.21.15.114:0/3872614343 wait complete. 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ERROR 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner: 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:ERROR: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:28:11.486 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 7881, in test_malicious_metafile_on_legacy_to_v1_upgrade 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner: subvol1, subvol2 = self._generate_random_subvolume_name(2) 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:AttributeError: 'TestMisc' object has no attribute '_generate_random_subvolume_name' 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner: 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:Ran 3 tests in 198.316s 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner: 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner: 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:ERROR: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-24T06:28:11.487 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:28:11.488 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:28:11.488 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 7881, in test_malicious_metafile_on_legacy_to_v1_upgrade 2023-11-24T06:28:11.488 INFO:tasks.cephfs_test_runner: subvol1, subvol2 = self._generate_random_subvolume_name(2) 2023-11-24T06:28:11.490 INFO:tasks.cephfs_test_runner:AttributeError: 'TestMisc' object has no attribute '_generate_random_subvolume_name' 2023-11-24T06:28:11.490 INFO:tasks.cephfs_test_runner: 2023-11-24T06:28:11.490 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:28:11.606 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3617ea8c90484640870736dfa88f7af8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-24T06:28:11.607 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:28:11.685 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-24T06:28:11.694 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T06:28:11.694 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:28:11.694 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:11.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:28:11.728 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2023-11-24T06:28:11.728 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:11.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:28:11.786 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:28:11.786 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:11.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:28:11.808 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2023-11-24T06:28:11.808 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:11.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=41,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3764 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:28:11.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.869 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:28:11.887 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:28:11.887 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:28:11.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:11.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:11.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:12.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2023-11-24T06:28:12.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.149 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2023-11-24T06:28:12.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.167 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2023-11-24T06:28:12.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:12.239 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2023-11-24T06:28:12.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.239 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:28:12.260 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:28:12.261 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:28:12.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:12.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:12.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.460 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-11-24T06:28:12.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.557 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2023-11-24T06:28:12.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:12.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2023-11-24T06:28:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2023-11-24T06:28:12.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:12.645 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:28:12.655 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:28:12.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:12.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.778+0000 7f4f35123700 1 Processor -- start 2023-11-24T06:28:12.780 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.778+0000 7f4f35123700 1 -- start start 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f35123700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f3006df20 0x7f4f3006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f35123700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3006ee70 con 0x7f4f3015ffb0 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f35123700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3006d4d0 con 0x7f4f30145760 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f35123700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f300cc5e0 con 0x7f4f3006df20 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2e59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f3006df20 0x7f4f3006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:12.783 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2f59e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4f3015ffb0 0x7f4f300cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58518/0 (socket says 172.21.15.114:58518) 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2e59c700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4f3006df20 0x7f4f3006e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42810/0 (socket says 172.21.15.114:42810) 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2f59e700 1 -- 172.21.15.114:0/32690609 learned_addr learned my addr 172.21.15.114:0/32690609 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2e59c700 1 -- 172.21.15.114:0/32690609 >> v1:172.21.15.172:6789/0 conn(0x7f4f30145760 legacy=0x7f4f3015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2e59c700 1 -- 172.21.15.114:0/32690609 >> v1:172.21.15.114:6789/0 conn(0x7f4f3015ffb0 legacy=0x7f4f300cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2e59c700 1 -- 172.21.15.114:0/32690609 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300cc8c0 con 0x7f4f3006df20 2023-11-24T06:28:12.784 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.779+0000 7f4f2dd9b700 1 -- 172.21.15.114:0/32690609 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825352936 0 0) 0x7f4f3006ee70 con 0x7f4f3015ffb0 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f2e59c700 1 --2- 172.21.15.114:0/32690609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3006e930 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009840 tx=0x7f4f24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af142141e8fafd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f2dd9b700 1 -- 172.21.15.114:0/32690609 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24011830 con 0x7f4f3006df20 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f2dd9b700 1 -- 172.21.15.114:0/32690609 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24011990 con 0x7f4f3006df20 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f2dd9b700 1 -- 172.21.15.114:0/32690609 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24011d20 con 0x7f4f3006df20 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 -- 172.21.15.114:0/32690609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 msgr2=0x7f4f3006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 --2- 172.21.15.114:0/32690609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3006e930 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009840 tx=0x7f4f24009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:12.785 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 -- 172.21.15.114:0/32690609 shutdown_connections 2023-11-24T06:28:12.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 --2- 172.21.15.114:0/32690609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3006e930 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 -- 172.21.15.114:0/32690609 >> 172.21.15.114:0/32690609 conn(0x7f4f300b1740 msgr2=0x7f4f300b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:12.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 -- 172.21.15.114:0/32690609 shutdown_connections 2023-11-24T06:28:12.786 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.780+0000 7f4f35123700 1 -- 172.21.15.114:0/32690609 wait complete. 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.782+0000 7f4f35123700 1 Processor -- start 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.782+0000 7f4f35123700 1 -- start start 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f30145760 0x7f4f3013be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 0x7f4f3015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3013c350 con 0x7f4f3006df20 2023-11-24T06:28:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3015b4c0 con 0x7f4f30145760 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3013c570 con 0x7f4f3015ffb0 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42814/0 (socket says 172.21.15.114:42814) 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 -- 172.21.15.114:0/1219174456 learned_addr learned my addr 172.21.15.114:0/1219174456 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2e59c700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f30145760 0x7f4f3013be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2f59e700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 0x7f4f3015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:12.790 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 -- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 msgr2=0x7f4f3015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 0x7f4f3015af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 -- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f30145760 msgr2=0x7f4f3013be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f30145760 0x7f4f3013be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 -- 172.21.15.114:0/1219174456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240094a0 con 0x7f4f3006df20 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2f59e700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 0x7f4f3015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f2ed9d700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c4e0 tx=0x7f4f2000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:12.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200052d0 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20005430 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3015d600 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200038c0 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.783+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3015dbc0 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.784+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f300751e0 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.785+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4f20003a20 con 0x7f4f3006df20 2023-11-24T06:28:12.792 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.785+0000 7f4f177fe700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 0x7f4f1c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.785+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4f200889f0 con 0x7f4f3006df20 2023-11-24T06:28:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.785+0000 7f4f2e59c700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 0x7f4f1c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.786+0000 7f4f2e59c700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 0x7f4f1c06d570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009840 tx=0x7f4f2401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.790+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f20053b20 con 0x7f4f3006df20 2023-11-24T06:28:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.974+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f3006da00 con 0x7f4f1c06b0b0 2023-11-24T06:28:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.976+0000 7f4f177fe700 1 -- 172.21.15.114:0/1219174456 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50849 (secure 0 0 0) 0x7f4f24011c00 con 0x7f4f1c06b0b0 2023-11-24T06:28:12.979 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 msgr2=0x7f4f1c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 0x7f4f1c06d570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009840 tx=0x7f4f2401c040 comp rx=0 tx=0).stop 2023-11-24T06:28:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 msgr2=0x7f4f3013dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:12.981 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c4e0 tx=0x7f4f2000c7f0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 shutdown_connections 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f4f1c06b0b0 0x7f4f1c06d570 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f3015ffb0 0x7f4f3015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f3006df20 0x7f4f3013dfd0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 --2- 172.21.15.114:0/1219174456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f30145760 0x7f4f3013be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 >> 172.21.15.114:0/1219174456 conn(0x7f4f300b1740 msgr2=0x7f4f3015f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 shutdown_connections 2023-11-24T06:28:12.983 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:12.978+0000 7f4f35123700 1 -- 172.21.15.114:0/1219174456 wait complete. 2023-11-24T06:28:12.983 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:28:12.993 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2023-11-24T06:28:11.758084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222944,"kb_used_data":6784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751776,"statfs":{"total":773094113280,"available":772865818624,"internally_reserved":0,"allocated":6946816,"data_stored":3570516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4199,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001306"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T06:28:09.466478+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:28:09.466478+0000","last_peered":"2023-11-24T06:28:09.466478+0000","last_clean":"2023-11-24T06:28:09.466478+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:28:09.466478+0000","last_undegraded":"2023-11-24T06:28:09.466478+0000","last_fullsized":"2023-11-24T06:28:09.466478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":892249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.623}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.432},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":295878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92900000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71199999999999997,"15min":0.84499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.294,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":1.294},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":3.427}]},{"osd":2,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.411,"15min":1.411},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.345,"15min":1.345},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":3.5209999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.7730000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54800000000000004,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":1.379,"15min":1.379},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.50800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":890229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.497},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.439}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.312,"15min":0.312},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.436,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51300000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55700000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53900000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81200000000000006,"5min":1.028,"15min":1.028},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85899999999999999,"5min":1.01,"15min":1.01},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63300000000000001,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63200000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.004,"15min":1.004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":1,"15min":1},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.252},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34699999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.629}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.755}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:12.995 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:13.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.112+0000 7f8191207700 1 Processor -- start 2023-11-24T06:28:13.115 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.112+0000 7f8191207700 1 -- start start 2023-11-24T06:28:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f8191207700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 0x7f818c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f8191207700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f818c06c710 0x7f818c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f8191207700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0ae430 con 0x7f818c0af4d0 2023-11-24T06:28:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f8191207700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0af0b0 con 0x7f818c06c710 2023-11-24T06:28:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f8191207700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b05a0 con 0x7f818c15e080 2023-11-24T06:28:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.113+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 0x7f818c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f818c06c710 0x7f818c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818a59c700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f818c0af4d0 0x7f818c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:52126/0 (socket says 172.21.15.114:52126) 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f818c06c710 0x7f818c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.114:38484/0 (socket says 172.21.15.114:38484) 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818a59c700 1 -- 172.21.15.114:0/3446521217 learned_addr learned my addr 172.21.15.114:0/3446521217 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 0x7f818c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:37366/0 (socket says 172.21.15.114:37366) 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f8189d9b700 1 -- 172.21.15.114:0/3446521217 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793558122 0 0) 0x7f818c0ae430 con 0x7f818c0af4d0 2023-11-24T06:28:13.119 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f8189d9b700 1 -- 172.21.15.114:0/3446521217 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81740035b0 con 0x7f818c0af4d0 2023-11-24T06:28:13.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 -- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 msgr2=0x7f818c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 --2- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 0x7f818c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 -- 172.21.15.114:0/3446521217 >> v1:172.21.15.172:6789/0 conn(0x7f818c0af4d0 legacy=0x7f818c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 -- 172.21.15.114:0/3446521217 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0ae710 con 0x7f818c06c710 2023-11-24T06:28:13.120 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f818b59e700 1 --2- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c06c710 0x7f818c0afb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8184009dc0 tx=0x7f818400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aadc005977c8a2d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f8189d9b700 1 -- 172.21.15.114:0/3446521217 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400e070 con 0x7f818c06c710 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f8189d9b700 1 -- 172.21.15.114:0/3446521217 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400bae0 con 0x7f818c06c710 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.114+0000 7f8189d9b700 1 -- 172.21.15.114:0/3446521217 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400be90 con 0x7f818c06c710 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 -- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c06c710 msgr2=0x7f818c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 --2- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c06c710 0x7f818c0afb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8184009dc0 tx=0x7f818400b330 comp rx=0 tx=0).stop 2023-11-24T06:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 -- 172.21.15.114:0/3446521217 shutdown_connections 2023-11-24T06:28:13.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 --2- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c06c710 0x7f818c0afb90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 --2- 172.21.15.114:0/3446521217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c15e080 0x7f818c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 -- 172.21.15.114:0/3446521217 >> 172.21.15.114:0/3446521217 conn(0x7f818c071b90 msgr2=0x7f818c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.122 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 -- 172.21.15.114:0/3446521217 shutdown_connections 2023-11-24T06:28:13.123 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.115+0000 7f8191207700 1 -- 172.21.15.114:0/3446521217 wait complete. 2023-11-24T06:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.116+0000 7f8191207700 1 Processor -- start 2023-11-24T06:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.116+0000 7f8191207700 1 -- start start 2023-11-24T06:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c0af4d0 0x7f818c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c15e080 0x7f818c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c1551c0 con 0x7f818c06c710 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c154330 con 0x7f818c15e080 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f8191207700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c1545b0 con 0x7f818c0af4d0 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42828/0 (socket says 172.21.15.114:42828) 2023-11-24T06:28:13.127 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f818ad9d700 1 -- 172.21.15.114:0/3293419317 learned_addr learned my addr 172.21.15.114:0/3293419317 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:13.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f818a59c700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c0af4d0 0x7f818c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.128 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.117+0000 7f818b59e700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c15e080 0x7f818c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 -- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c0af4d0 msgr2=0x7f818c159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c0af4d0 0x7f818c159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 -- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c15e080 msgr2=0x7f818c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c15e080 0x7f818c153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 -- 172.21.15.114:0/3293419317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184009a70 con 0x7f818c06c710 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f818ad9d700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f818000e8a0 tx=0x7f818000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.130 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180009590 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8180013680 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800139f0 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c154890 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.118+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c154e50 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.119+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c0753f0 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.119+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8180013b50 con 0x7f818c06c710 2023-11-24T06:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.120+0000 7f81737fe700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 0x7f817806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.120+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8180089500 con 0x7f818c06c710 2023-11-24T06:28:13.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.120+0000 7f818a59c700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 0x7f817806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.120+0000 7f818a59c700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 0x7f817806d540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f818c153da0 tx=0x7f817c007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.132 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.124+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8180054590 con 0x7f818c06c710 2023-11-24T06:28:13.315 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.313+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f818c0aecd0 con 0x7f817806b080 2023-11-24T06:28:13.317 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.315+0000 7f81737fe700 1 -- 172.21.15.114:0/3293419317 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50849 (secure 0 0 0) 0x7f817c001370 con 0x7f817806b080 2023-11-24T06:28:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:13.319 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 msgr2=0x7f817806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 0x7f817806d540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f818c153da0 tx=0x7f817c007dc0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 msgr2=0x7f818c155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f818000e8a0 tx=0x7f818000ebb0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 shutdown_connections 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f817806b080 0x7f817806d540 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c15e080 0x7f818c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.320 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818c06c710 0x7f818c155650 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 --2- 172.21.15.114:0/3293419317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f818c0af4d0 0x7f818c159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 >> 172.21.15.114:0/3293419317 conn(0x7f818c071b90 msgr2=0x7f818c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 shutdown_connections 2023-11-24T06:28:13.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.317+0000 7f8191207700 1 -- 172.21.15.114:0/3293419317 wait complete. 2023-11-24T06:28:13.321 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:28:13.331 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2023-11-24T06:28:11.758084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222944,"kb_used_data":6784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751776,"statfs":{"total":773094113280,"available":772865818624,"internally_reserved":0,"allocated":6946816,"data_stored":3570516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4199,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001306"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T06:28:09.466478+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:28:09.466478+0000","last_peered":"2023-11-24T06:28:09.466478+0000","last_clean":"2023-11-24T06:28:09.466478+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:28:09.466478+0000","last_undegraded":"2023-11-24T06:28:09.466478+0000","last_fullsized":"2023-11-24T06:28:09.466478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":892249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.623}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.432},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":295878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92900000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71199999999999997,"15min":0.84499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.294,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":1.294},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":3.427}]},{"osd":2,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.411,"15min":1.411},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.345,"15min":1.345},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":3.5209999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.7730000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54800000000000004,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":1.379,"15min":1.379},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.50800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":890229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.497},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.439}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.312,"15min":0.312},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.436,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51300000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55700000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53900000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81200000000000006,"5min":1.028,"15min":1.028},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85899999999999999,"5min":1.01,"15min":1.01},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63300000000000001,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63200000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.004,"15min":1.004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":1,"15min":1},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.252},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34699999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.629}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.755}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:13.333 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:28:13.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.450+0000 7f6f9b934700 1 Processor -- start 2023-11-24T06:28:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 -- start start 2023-11-24T06:28:13.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f9415e080 0x7f6f940b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f940ae430 con 0x7f6f9406c000 2023-11-24T06:28:13.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f940af0b0 con 0x7f6f9415e080 2023-11-24T06:28:13.454 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.451+0000 7f6f9b934700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940b05a0 con 0x7f6f940af4d0 2023-11-24T06:28:13.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f996d0700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6f9415e080 0x7f6f940b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:37380/0 (socket says 172.21.15.114:37380) 2023-11-24T06:28:13.456 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 -- 172.21.15.114:0/2630222679 learned_addr learned my addr 172.21.15.114:0/2630222679 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 -- 172.21.15.114:0/2630222679 >> v2:172.21.15.172:3300/0 conn(0x7f6f9415e080 msgr2=0x7f6f940b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 --2- 172.21.15.114:0/2630222679 >> v2:172.21.15.172:3300/0 conn(0x7f6f9415e080 0x7f6f940b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 -- 172.21.15.114:0/2630222679 >> v1:172.21.15.172:6789/0 conn(0x7f6f9406c000 legacy=0x7f6f94073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 -- 172.21.15.114:0/2630222679 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f940ae710 con 0x7f6f940af4d0 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.452+0000 7f6f98ecf700 1 --2- 172.21.15.114:0/2630222679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f900098f0 tx=0x7f6f90009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638d1fbb7e09cd0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f8bfff700 1 -- 172.21.15.114:0/2630222679 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011830 con 0x7f6f940af4d0 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f8bfff700 1 -- 172.21.15.114:0/2630222679 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f90011990 con 0x7f6f940af4d0 2023-11-24T06:28:13.457 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f8bfff700 1 -- 172.21.15.114:0/2630222679 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90011d20 con 0x7f6f940af4d0 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 -- 172.21.15.114:0/2630222679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 msgr2=0x7f6f940afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 --2- 172.21.15.114:0/2630222679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f900098f0 tx=0x7f6f90009c00 comp rx=0 tx=0).stop 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 -- 172.21.15.114:0/2630222679 shutdown_connections 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 --2- 172.21.15.114:0/2630222679 >> v2:172.21.15.172:3300/0 conn(0x7f6f9415e080 0x7f6f940b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 --2- 172.21.15.114:0/2630222679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f940af4d0 0x7f6f940afb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 -- 172.21.15.114:0/2630222679 >> 172.21.15.114:0/2630222679 conn(0x7f6f94071b90 msgr2=0x7f6f94071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 -- 172.21.15.114:0/2630222679 shutdown_connections 2023-11-24T06:28:13.458 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.453+0000 7f6f9b934700 1 -- 172.21.15.114:0/2630222679 wait complete. 2023-11-24T06:28:13.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.454+0000 7f6f9b934700 1 Processor -- start 2023-11-24T06:28:13.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.454+0000 7f6f9b934700 1 -- start start 2023-11-24T06:28:13.461 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940af4d0 0x7f6f94157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 0x7f6f94159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94154eb0 con 0x7f6f9406c000 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94158860 con 0x7f6f940af4d0 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f9b934700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94146a00 con 0x7f6f9415e080 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f996d0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f996d0700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42842/0 (socket says 172.21.15.114:42842) 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f996d0700 1 -- 172.21.15.114:0/1308301042 learned_addr learned my addr 172.21.15.114:0/1308301042 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f98ecf700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940af4d0 0x7f6f94157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.455+0000 7f6f99ed1700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 0x7f6f94159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 -- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 msgr2=0x7f6f94159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 0x7f6f94159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 -- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940af4d0 msgr2=0x7f6f94157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940af4d0 0x7f6f94157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 -- 172.21.15.114:0/1308301042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90009550 con 0x7f6f9406c000 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f99ed1700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 0x7f6f94159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f996d0700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e8a0 tx=0x7f6f8400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84009590 con 0x7f6f9406c000 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84013680 con 0x7f6f9406c000 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840139f0 con 0x7f6f9406c000 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f94158580 con 0x7f6f9406c000 2023-11-24T06:28:13.464 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.456+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f94155370 con 0x7f6f9406c000 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.457+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f940753f0 con 0x7f6f9406c000 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.457+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6f84013b50 con 0x7f6f9406c000 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.458+0000 7f6f89ffb700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 0x7f6f7806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.458+0000 7f6f98ecf700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 0x7f6f7806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.458+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f84089540 con 0x7f6f9406c000 2023-11-24T06:28:13.465 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.458+0000 7f6f98ecf700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 0x7f6f7806d570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009c40 tx=0x7f6f90008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.463+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f840545e0 con 0x7f6f9406c000 2023-11-24T06:28:13.653 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.650+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f940aecd0 con 0x7f6f7806b0b0 2023-11-24T06:28:13.655 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.653+0000 7f6f89ffb700 1 -- 172.21.15.114:0/1308301042 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50849 (secure 0 0 0) 0x7f6f90011b00 con 0x7f6f7806b0b0 2023-11-24T06:28:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:13.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 msgr2=0x7f6f7806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 0x7f6f7806d570 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009c40 tx=0x7f6f90008030 comp rx=0 tx=0).stop 2023-11-24T06:28:13.657 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 msgr2=0x7f6f941464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400e8a0 tx=0x7f6f8400ebb0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 shutdown_connections 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f6f7806b0b0 0x7f6f7806d570 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6f9415e080 0x7f6f94159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f9406c000 0x7f6f941464c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 --2- 172.21.15.114:0/1308301042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f940af4d0 0x7f6f94157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 >> 172.21.15.114:0/1308301042 conn(0x7f6f94071b90 msgr2=0x7f6f940d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.658 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 shutdown_connections 2023-11-24T06:28:13.659 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.655+0000 7f6f9b934700 1 -- 172.21.15.114:0/1308301042 wait complete. 2023-11-24T06:28:13.659 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:28:13.669 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2023-11-24T06:28:11.758084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222944,"kb_used_data":6784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751776,"statfs":{"total":773094113280,"available":772865818624,"internally_reserved":0,"allocated":6946816,"data_stored":3570516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4199,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001306"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T06:28:09.466478+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:28:09.466478+0000","last_peered":"2023-11-24T06:28:09.466478+0000","last_clean":"2023-11-24T06:28:09.466478+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:28:09.466478+0000","last_undegraded":"2023-11-24T06:28:09.466478+0000","last_fullsized":"2023-11-24T06:28:09.466478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":892249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79200000000000004,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69099999999999995,"15min":0.74299999999999999},"min":{"1min":0.495,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74199999999999999,"5min":4.407,"15min":4.407},"last":0.623}]},{"osd":1,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.82899999999999996,"15min":0.94199999999999995},"min":{"1min":0.47099999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73099999999999998,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.67100000000000004,"5min":1.232,"15min":1.232},"last":0.432},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.81899999999999995,"15min":0.93700000000000006},"min":{"1min":0.42699999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70999999999999996,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.621},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69999999999999996,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76900000000000002,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65300000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72899999999999998,"5min":1.131,"15min":1.131},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.64400000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.75700000000000001,"5min":1.079,"15min":1.079},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.63700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":1.042,"15min":1.042},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71499999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":295878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92900000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71199999999999997,"15min":0.84499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.294,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":1.294},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":3.427}]},{"osd":2,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.411,"15min":1.411},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.345,"15min":1.345},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":3.5209999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.7730000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54800000000000004,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":1.379,"15min":1.379},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.50800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.246,"5min":0.188,"15min":0.188},"max":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.191},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.374},"min":{"1min":0.29599999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.375,"15min":0.371},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57799999999999996,"5min":0.75,"15min":0.75},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85199999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.504,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82499999999999996,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82799999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.77900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":890229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.81000000000000005,"15min":1.02},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51800000000000002,"5min":11.44,"15min":11.44},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.51100000000000001,"15min":0.57799999999999996},"min":{"1min":0.27400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.59099999999999997,"15min":0.66700000000000004},"min":{"1min":0.32600000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54700000000000004,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.84299999999999997,"15min":1.0449999999999999},"min":{"1min":0.33900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.56200000000000006,"5min":11.157,"15min":11.157},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.59399999999999997,"15min":0.66700000000000004},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.621,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.52800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58299999999999996,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":1.0720000000000001,"15min":1.286},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93799999999999994,"5min":11.378,"15min":11.378},"last":0.497},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.75800000000000001,"15min":0.83199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.439}]},{"osd":5,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.42599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95599999999999996,"5min":1.728,"15min":1.728},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.81699999999999995,"15min":0.89600000000000002},"min":{"1min":0.48999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.72499999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.7,"15min":1.7},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.089,"15min":1.3049999999999999},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91900000000000004,"5min":11.343,"15min":11.343},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:27:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.1279999999999999,"15min":1.339},"min":{"1min":0.501,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98599999999999999,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.48599999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.84399999999999997,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.312,"15min":0.312},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.436,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51300000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55700000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53900000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81200000000000006,"5min":1.028,"15min":1.028},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85899999999999999,"5min":1.01,"15min":1.01},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63300000000000001,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63200000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.004,"15min":1.004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":1,"15min":1},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":295934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.252},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.627},"min":{"1min":0.37,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.75,"5min":2.169,"15min":2.169},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.433,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.40400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.45700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71799999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":1.351,"15min":1.351},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34699999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.83099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.629}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.755}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:13.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:13.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.789+0000 7f5eba470700 1 Processor -- start 2023-11-24T06:28:13.791 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.789+0000 7f5eba470700 1 -- start start 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eba470700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5eb41446a0 0x7f5eb40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eba470700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 0x7f5eb40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eba470700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb40af950 con 0x7f5eb40adf00 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eba470700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb4144a80 con 0x7f5eb41446a0 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eba470700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb40afc30 con 0x7f5eb406c710 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5eb41446a0 0x7f5eb40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5eb41446a0 0x7f5eb40b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42850/0 (socket says 172.21.15.114:42850) 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 -- 172.21.15.114:0/2576923077 learned_addr learned my addr 172.21.15.114:0/2576923077 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb8a0d700 1 --2- 172.21.15.114:0/2576923077 >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 0x7f5eb40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.795 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 -- 172.21.15.114:0/2576923077 >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 msgr2=0x7f5eb40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 --2- 172.21.15.114:0/2576923077 >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 0x7f5eb40b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 -- 172.21.15.114:0/2576923077 >> v1:172.21.15.172:6789/0 conn(0x7f5eb40adf00 legacy=0x7f5eb406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 -- 172.21.15.114:0/2576923077 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40aff10 con 0x7f5eb41446a0 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb8a0d700 1 --2- 172.21.15.114:0/2576923077 >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 0x7f5eb40b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb3fff700 1 --2- 172.21.15.114:0/2576923077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb40b5ae0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40aedf0 tx=0x7f5ea400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d27aa1839821424 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb2ffd700 1 -- 172.21.15.114:0/2576923077 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400dcf0 con 0x7f5eb41446a0 2023-11-24T06:28:13.796 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb2ffd700 1 -- 172.21.15.114:0/2576923077 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea400de50 con 0x7f5eb41446a0 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.790+0000 7f5eb2ffd700 1 -- 172.21.15.114:0/2576923077 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4004250 con 0x7f5eb41446a0 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 -- 172.21.15.114:0/2576923077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 msgr2=0x7f5eb40b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 --2- 172.21.15.114:0/2576923077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb40b5ae0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40aedf0 tx=0x7f5ea400be70 comp rx=0 tx=0).stop 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 -- 172.21.15.114:0/2576923077 shutdown_connections 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 --2- 172.21.15.114:0/2576923077 >> v2:172.21.15.172:3300/0 conn(0x7f5eb406c710 0x7f5eb40b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 --2- 172.21.15.114:0/2576923077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb40b5ae0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 -- 172.21.15.114:0/2576923077 >> 172.21.15.114:0/2576923077 conn(0x7f5eb4071b90 msgr2=0x7f5eb4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 -- 172.21.15.114:0/2576923077 shutdown_connections 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.791+0000 7f5eba470700 1 -- 172.21.15.114:0/2576923077 wait complete. 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.792+0000 7f5eba470700 1 Processor -- start 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.792+0000 7f5eba470700 1 -- start start 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5eb406c710 0x7f5eb409da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb40adf00 0x7f5eb409dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.802 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb40d0070 con 0x7f5eb41446a0 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41496f0 con 0x7f5eb40adf00 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eba470700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb41499d0 con 0x7f5eb406c710 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42860/0 (socket says 172.21.15.114:42860) 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 -- 172.21.15.114:0/3819764091 learned_addr learned my addr 172.21.15.114:0/3819764091 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb3fff700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5eb406c710 0x7f5eb409da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb37fe700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb40adf00 0x7f5eb409dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.803 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 -- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5eb406c710 msgr2=0x7f5eb409da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5eb406c710 0x7f5eb409da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 -- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb40adf00 msgr2=0x7f5eb409dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb40adf00 0x7f5eb409dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.793+0000 7f5eb8a0d700 1 -- 172.21.15.114:0/3819764091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea400b7f0 con 0x7f5eb41446a0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eb8a0d700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00ce80 tx=0x7f5eac00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00ad40 con 0x7f5eb41446a0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac015440 con 0x7f5eb41446a0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb4149cb0 con 0x7f5eb41446a0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00aea0 con 0x7f5eb41446a0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.794+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb409b0d0 con 0x7f5eb41446a0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.795+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb40753f0 con 0x7f5eb41446a0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.795+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5eac00d430 con 0x7f5eb41446a0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.796+0000 7f5eb0ff9700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 0x7f5ea006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.796+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5eac0899c0 con 0x7f5eb41446a0 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.796+0000 7f5eb3fff700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 0x7f5ea006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.796+0000 7f5eb3fff700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 0x7f5ea006d540 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000c00 tx=0x7f5ea401d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.800+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5eac054b80 con 0x7f5eb41446a0 2023-11-24T06:28:13.996 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.994+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eb414a1d0 con 0x7f5eb41446a0 2023-11-24T06:28:13.997 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.994+0000 7f5eb0ff9700 1 -- 172.21.15.114:0/3819764091 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13024 (secure 0 0 0) 0x7f5eac01a020 con 0x7f5eb41446a0 2023-11-24T06:28:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:13.997 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":61,"fsid":"cfb0acb2-02f2-43f9-bba4-c35657dcf6fd","created":"2023-11-24T06:23:52.085426+0000","modified":"2023-11-24T06:28:10.458427+0000","last_up_change":"2023-11-24T06:24:03.735016+0000","last_in_change":"2023-11-24T06:23:54.578009+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:24:05.750313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc893cbd-20b7-40ac-ae45-33325e2aaf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6809","nonce":1249744364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6811","nonce":1249744364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6815","nonce":1249744364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1249744364},{"type":"v1","addr":"172.21.15.114:6813","nonce":1249744364}]},"public_addr":"172.21.15.114:6809/1249744364","cluster_addr":"172.21.15.114:6811/1249744364","heartbeat_back_addr":"172.21.15.114:6815/1249744364","heartbeat_front_addr":"172.21.15.114:6813/1249744364","state":["exists","up"]},{"osd":1,"uuid":"ce10f334-6cb1-4d9b-9cea-d54523268a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6801","nonce":2279805148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6803","nonce":2279805148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6807","nonce":2279805148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2279805148},{"type":"v1","addr":"172.21.15.114:6805","nonce":2279805148}]},"public_addr":"172.21.15.114:6801/2279805148","cluster_addr":"172.21.15.114:6803/2279805148","heartbeat_back_addr":"172.21.15.114:6807/2279805148","heartbeat_front_addr":"172.21.15.114:6805/2279805148","state":["exists","up"]},{"osd":2,"uuid":"7823a8b2-097f-44dc-9ddb-0f979186b3d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6817","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6820","nonce":2005258798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6825","nonce":2005258798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6831","nonce":2005258798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6827","nonce":2005258798},{"type":"v1","addr":"172.21.15.114:6829","nonce":2005258798}]},"public_addr":"172.21.15.114:6820/2005258798","cluster_addr":"172.21.15.114:6825/2005258798","heartbeat_back_addr":"172.21.15.114:6831/2005258798","heartbeat_front_addr":"172.21.15.114:6829/2005258798","state":["exists","up"]},{"osd":3,"uuid":"140a162c-f0b9-4d80-9c96-08b6a2f01f43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6818","nonce":783795072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6819","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6821","nonce":783795072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6828","nonce":783795072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":783795072},{"type":"v1","addr":"172.21.15.114:6824","nonce":783795072}]},"public_addr":"172.21.15.114:6818/783795072","cluster_addr":"172.21.15.114:6821/783795072","heartbeat_back_addr":"172.21.15.114:6828/783795072","heartbeat_front_addr":"172.21.15.114:6824/783795072","state":["exists","up"]},{"osd":4,"uuid":"6bdec982-f0c6-4d00-b5e2-8d5cdceeb597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6801","nonce":517626883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6803","nonce":517626883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6807","nonce":517626883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":517626883},{"type":"v1","addr":"172.21.15.172:6805","nonce":517626883}]},"public_addr":"172.21.15.172:6801/517626883","cluster_addr":"172.21.15.172:6803/517626883","heartbeat_back_addr":"172.21.15.172:6807/517626883","heartbeat_front_addr":"172.21.15.172:6805/517626883","state":["exists","up"]},{"osd":5,"uuid":"fb61e375-0fe3-488a-9b10-fab492072d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6817","nonce":1185079648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6819","nonce":1185079648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6827","nonce":1185079648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1185079648},{"type":"v1","addr":"172.21.15.172:6822","nonce":1185079648}]},"public_addr":"172.21.15.172:6817/1185079648","cluster_addr":"172.21.15.172:6819/1185079648","heartbeat_back_addr":"172.21.15.172:6827/1185079648","heartbeat_front_addr":"172.21.15.172:6822/1185079648","state":["exists","up"]},{"osd":6,"uuid":"4f66bf03-958c-4802-b95d-f883e98831cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6823","nonce":1378310518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6826","nonce":1378310518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6831","nonce":1378310518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1378310518},{"type":"v1","addr":"172.21.15.172:6829","nonce":1378310518}]},"public_addr":"172.21.15.172:6823/1378310518","cluster_addr":"172.21.15.172:6826/1378310518","heartbeat_back_addr":"172.21.15.172:6831/1378310518","heartbeat_front_addr":"172.21.15.172:6829/1378310518","state":["exists","up"]},{"osd":7,"uuid":"b1be8744-fdb3-40b7-825b-492a3375649d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6809","nonce":2370047192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6811","nonce":2370047192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6815","nonce":2370047192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2370047192},{"type":"v1","addr":"172.21.15.172:6813","nonce":2370047192}]},"public_addr":"172.21.15.172:6809/2370047192","cluster_addr":"172.21.15.172:6811/2370047192","heartbeat_back_addr":"172.21.15.172:6815/2370047192","heartbeat_front_addr":"172.21.15.172:6813/2370047192","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.814420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.712703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.837439+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.808220+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.771860+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.743373+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.775492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:23:59.797194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6837/3988687339":"2023-11-25T06:28:07.450328+0000","172.21.15.114:6835/3988687339":"2023-11-25T06:28:07.450328+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:13.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.996+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 msgr2=0x7f5ea006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.996+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 0x7f5ea006d540 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000c00 tx=0x7f5ea401d040 comp rx=0 tx=0).stop 2023-11-24T06:28:13.999 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.996+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 msgr2=0x7f5eb414b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.996+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00ce80 tx=0x7f5eac00ceb0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.996+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 shutdown_connections 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f5ea006b080 0x7f5ea006d540 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb41446a0 0x7f5eb414b650 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5eb406c710 0x7f5eb409da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 --2- 172.21.15.114:0/3819764091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb40adf00 0x7f5eb409dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 >> 172.21.15.114:0/3819764091 conn(0x7f5eb4071b90 msgr2=0x7f5eb40b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.000 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 shutdown_connections 2023-11-24T06:28:14.001 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:13.997+0000 7f5eba470700 1 -- 172.21.15.114:0/3819764091 wait complete. 2023-11-24T06:28:15.012 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:28:15.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:15.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.162+0000 7f22c432d700 1 Processor -- start 2023-11-24T06:28:15.165 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.162+0000 7f22c432d700 1 -- start start 2023-11-24T06:28:15.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c432d700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22bc06f480 0x7f22bc06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c432d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc06b200 con 0x7f22bc06e830 2023-11-24T06:28:15.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c432d700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc06b4e0 con 0x7f22bc144600 2023-11-24T06:28:15.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c432d700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc06aff0 con 0x7f22bc06f480 2023-11-24T06:28:15.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c18c8700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22bc06f480 0x7f22bc06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c28ca700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f22bc06e830 0x7f22bc0cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58576/0 (socket says 172.21.15.114:58576) 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.163+0000 7f22c28ca700 1 -- 172.21.15.114:0/242606429 learned_addr learned my addr 172.21.15.114:0/242606429 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c18c8700 1 -- 172.21.15.114:0/242606429 >> v1:172.21.15.172:6789/0 conn(0x7f22bc144600 legacy=0x7f22bc0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c18c8700 1 -- 172.21.15.114:0/242606429 >> v1:172.21.15.114:6789/0 conn(0x7f22bc06e830 legacy=0x7f22bc0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c10c7700 1 -- 172.21.15.114:0/242606429 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786121395 0 0) 0x7f22bc06b200 con 0x7f22bc06e830 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c18c8700 1 -- 172.21.15.114:0/242606429 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc06b7c0 con 0x7f22bc06f480 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c18c8700 1 --2- 172.21.15.114:0/242606429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06f480 0x7f22bc06c2a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009840 tx=0x7f22b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1652701ba332d2ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c10c7700 1 -- 172.21.15.114:0/242606429 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8011830 con 0x7f22bc06f480 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c10c7700 1 -- 172.21.15.114:0/242606429 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8011990 con 0x7f22bc06f480 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c10c7700 1 -- 172.21.15.114:0/242606429 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8011d20 con 0x7f22bc06f480 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c432d700 1 -- 172.21.15.114:0/242606429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06f480 msgr2=0x7f22bc06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.164+0000 7f22c432d700 1 --2- 172.21.15.114:0/242606429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06f480 0x7f22bc06c2a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009840 tx=0x7f22b8009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.165+0000 7f22c432d700 1 -- 172.21.15.114:0/242606429 shutdown_connections 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.165+0000 7f22c432d700 1 --2- 172.21.15.114:0/242606429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06f480 0x7f22bc06c2a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.165+0000 7f22c432d700 1 -- 172.21.15.114:0/242606429 >> 172.21.15.114:0/242606429 conn(0x7f22bc0d0b40 msgr2=0x7f22bc0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.165+0000 7f22c432d700 1 -- 172.21.15.114:0/242606429 shutdown_connections 2023-11-24T06:28:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.165+0000 7f22c432d700 1 -- 172.21.15.114:0/242606429 wait complete. 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 Processor -- start 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 -- start start 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 0x7f22bc13e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22bc06f480 0x7f22bc15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc144600 0x7f22bc15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc13d380 con 0x7f22bc06e830 2023-11-24T06:28:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc13d660 con 0x7f22bc144600 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.166+0000 7f22c432d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc13d940 con 0x7f22bc06f480 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 0x7f22bc13e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 0x7f22bc13e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42870/0 (socket says 172.21.15.114:42870) 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 -- 172.21.15.114:0/2213599291 learned_addr learned my addr 172.21.15.114:0/2213599291 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c28ca700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc144600 0x7f22bc15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c18c8700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22bc06f480 0x7f22bc15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 -- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22bc06f480 msgr2=0x7f22bc15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22bc06f480 0x7f22bc15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 -- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc144600 msgr2=0x7f22bc15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc144600 0x7f22bc15b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80094a0 con 0x7f22bc06e830 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c28ca700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc144600 0x7f22bc15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c20c9700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 0x7f22bc13e420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c450 tx=0x7f22ac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0052d0 con 0x7f22bc06e830 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac005430 con 0x7f22bc06e830 2023-11-24T06:28:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0038c0 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc13dc20 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.167+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc15ea10 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.168+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22bc15ebd0 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f22ac003a20 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22a006b080 0x7f22a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f22ac088c90 con 0x7f22bc06e830 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22c18c8700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22a006b080 0x7f22a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7f22a0070ee0 0x7f22a0073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22a00739d0 con 0x7f22a0070ee0 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ac0890c0 con 0x7f22bc06e830 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22c28ca700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7f22a0070ee0 0x7f22a0073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.169+0000 7f22c18c8700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22a006b080 0x7f22a006d540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22bc15d0c0 tx=0x7f22b801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.170+0000 7f22c28ca700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7f22a0070ee0 0x7f22a0073300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.170+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== osd.0 v2:172.21.15.114:6808/1249744364 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f22a00739d0 con 0x7f22a0070ee0 2023-11-24T06:28:15.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.185+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 --> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f22bc1449e0 con 0x7f22a0070ee0 2023-11-24T06:28:15.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.185+0000 7f22b2ffd700 1 -- 172.21.15.114:0/2213599291 <== osd.0 v2:172.21.15.114:6808/1249744364 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f22b4007230 con 0x7f22a0070ee0 2023-11-24T06:28:15.188 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:15.188 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:15.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7f22a0070ee0 msgr2=0x7f22a0073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6808/1249744364,v1:172.21.15.114:6809/1249744364] conn(0x7f22a0070ee0 0x7f22a0073300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22a006b080 msgr2=0x7f22a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f22a006b080 0x7f22a006d540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22bc15d0c0 tx=0x7f22b801c040 comp rx=0 tx=0).stop 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 msgr2=0x7f22bc13e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 --2- 172.21.15.114:0/2213599291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22bc06e830 0x7f22bc13e420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c450 tx=0x7f22ac00c810 comp rx=0 tx=0).stop 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c28ca700 1 -- 172.21.15.114:0/2213599291 reap_dead start 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 shutdown_connections 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 >> 172.21.15.114:0/2213599291 conn(0x7f22bc0d0b40 msgr2=0x7f22bc143760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 shutdown_connections 2023-11-24T06:28:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.186+0000 7f22c432d700 1 -- 172.21.15.114:0/2213599291 wait complete. 2023-11-24T06:28:15.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:28:15.321 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.319+0000 7f9eb558c700 1 Processor -- start 2023-11-24T06:28:15.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.319+0000 7f9eb558c700 1 -- start start 2023-11-24T06:28:15.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eb558c700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9eb00b2b60 0x7f9eb00ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eb558c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb00cf0d0 0x7f9eb00d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eb558c700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb006bc00 con 0x7f9eb015e0f0 2023-11-24T06:28:15.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eb558c700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb00cf560 con 0x7f9eb00b2b60 2023-11-24T06:28:15.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eb558c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb015e4d0 con 0x7f9eb00cf0d0 2023-11-24T06:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eaeffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9eb015e0f0 0x7f9eb00b9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58588/0 (socket says 172.21.15.114:58588) 2023-11-24T06:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eaeffd700 1 -- 172.21.15.114:0/2897166690 learned_addr learned my addr 172.21.15.114:0/2897166690 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> v2:172.21.15.172:3300/0 conn(0x7f9eb00b2b60 0x7f9eb00ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eaf7fe700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb00cf0d0 0x7f9eb00d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056241145 0 0) 0x7f9eb006bc00 con 0x7f9eb015e0f0 2023-11-24T06:28:15.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e980035b0 con 0x7f9eb015e0f0 2023-11-24T06:28:15.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1894176261 0 0) 0x7f9e980035b0 con 0x7f9eb015e0f0 2023-11-24T06:28:15.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb00cf0d0 msgr2=0x7f9eb00d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb00cf0d0 0x7f9eb00d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 >> v2:172.21.15.172:3300/0 conn(0x7f9eb00b2b60 msgr2=0x7f9eb00ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 --2- 172.21.15.114:0/2897166690 >> v2:172.21.15.172:3300/0 conn(0x7f9eb00b2b60 0x7f9eb00ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb006aff0 con 0x7f9eb015e0f0 2023-11-24T06:28:15.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f9ea4001670 con 0x7f9eb015e0f0 2023-11-24T06:28:15.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 >> v1:172.21.15.114:6789/0 conn(0x7f9eb015e0f0 legacy=0x7f9eb00b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e98003a00 0x7f9e98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.320+0000 7f9eadffb700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9e980048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98004f40 0x7f9e9800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00cf560 con 0x7f9e980043f0 2023-11-24T06:28:15.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb015e4d0 con 0x7f9e98004f40 2023-11-24T06:28:15.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9800bf50 con 0x7f9e98003a00 2023-11-24T06:28:15.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eaf7fe700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb00cf0d0 0x7f9eb00d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:15.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> v2:172.21.15.172:3300/0 conn(0x7f9eb00b2b60 0x7f9eb00ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:15.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9e980048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eaeffd700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e98003a00 0x7f9e98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eaf7fe700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98004f40 0x7f9e9800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 -- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e98003a00 msgr2=0x7f9e98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e98003a00 0x7f9e98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 -- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98004f40 msgr2=0x7f9e9800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98004f40 0x7f9e9800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 -- 172.21.15.114:0/2897166690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980035b0 con 0x7f9e980043f0 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9e980048d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d10 tx=0x7f9e98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a3bd91d356ee91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0009e60 con 0x7f9e980043f0 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000e9a0 con 0x7f9e980043f0 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.321+0000 7f9eadffb700 1 -- 172.21.15.114:0/2897166690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000ed10 con 0x7f9e980043f0 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 -- 172.21.15.114:0/2897166690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 msgr2=0x7f9e980048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 --2- 172.21.15.114:0/2897166690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9e980048d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d10 tx=0x7f9e98005880 comp rx=0 tx=0).stop 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eaf7fe700 1 -- 172.21.15.114:0/2897166690 reap_dead start 2023-11-24T06:28:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 -- 172.21.15.114:0/2897166690 shutdown_connections 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 -- 172.21.15.114:0/2897166690 >> 172.21.15.114:0/2897166690 conn(0x7f9eb00b5ce0 msgr2=0x7f9eb00b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 -- 172.21.15.114:0/2897166690 shutdown_connections 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.322+0000 7f9eb558c700 1 -- 172.21.15.114:0/2897166690 wait complete. 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.323+0000 7f9eb558c700 1 Processor -- start 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 -- start start 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb01599c0 0x7f9eb0155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb0155e90 0x7f9eb01f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0156320 con 0x7f9e980043f0 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb015aa20 con 0x7f9eb01599c0 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eb558c700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0159ea0 con 0x7f9eb0155e90 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.324+0000 7f9eaeffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42892/0 (socket says 172.21.15.114:42892) 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 -- 172.21.15.114:0/392783542 learned_addr learned my addr 172.21.15.114:0/392783542 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 -- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb0155e90 msgr2=0x7f9eb01f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb0155e90 0x7f9eb01f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 -- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb01599c0 msgr2=0x7f9eb0155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb01599c0 0x7f9eb0155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 -- 172.21.15.114:0/392783542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb015a180 con 0x7f9e980043f0 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eaeffd700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40072f0 tx=0x7f9ea4007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400f660 con 0x7f9e980043f0 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea400f7e0 con 0x7f9e980043f0 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400fb90 con 0x7f9e980043f0 2023-11-24T06:28:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb015a460 con 0x7f9e980043f0 2023-11-24T06:28:15.345 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.325+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01554a0 con 0x7f9e980043f0 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.326+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb00751e0 con 0x7f9e980043f0 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.327+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9ea400fcf0 con 0x7f9e980043f0 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.327+0000 7f9e977fe700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 0x7f9e9c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.327+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ea4088c20 con 0x7f9e980043f0 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.327+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 0x7f9e9c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.346 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.327+0000 7f9eae7fc700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 0x7f9e9c06d540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9e980047d0 tx=0x7f9ea0013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.347 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.331+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ea4053d50 con 0x7f9e980043f0 2023-11-24T06:28:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.521+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9eb006b890 con 0x7f9e9c06b080 2023-11-24T06:28:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.522+0000 7f9e977fe700 1 -- 172.21.15.114:0/392783542 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ea000eb90 con 0x7f9e9c06b080 2023-11-24T06:28:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 msgr2=0x7f9e9c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 0x7f9e9c06d540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9e980047d0 tx=0x7f9ea0013000 comp rx=0 tx=0).stop 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 msgr2=0x7f9eb0154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40072f0 tx=0x7f9ea4007600 comp rx=0 tx=0).stop 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 shutdown_connections 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f9e9c06b080 0x7f9e9c06d540 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb0155e90 0x7f9eb01f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e980043f0 0x7f9eb0154f60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 --2- 172.21.15.114:0/392783542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb01599c0 0x7f9eb0155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 >> 172.21.15.114:0/392783542 conn(0x7f9eb00b5ce0 msgr2=0x7f9eb00cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 shutdown_connections 2023-11-24T06:28:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.524+0000 7f9eb558c700 1 -- 172.21.15.114:0/392783542 wait complete. 2023-11-24T06:28:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:28:15.538 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:28:15.538 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.659+0000 7fbf83cab700 1 Processor -- start 2023-11-24T06:28:15.662 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 -- start start 2023-11-24T06:28:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf7c073f90 0x7fbf7c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 0x7fbf7c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c06ee20 con 0x7fbf7c0b5850 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c1470a0 con 0x7fbf7c073f90 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.660+0000 7fbf83cab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b3b50 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf7c073f90 0x7fbf7c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81246700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf7c0b5850 0x7fbf7c074370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58608/0 (socket says 172.21.15.114:58608) 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81246700 1 -- 172.21.15.114:0/219568497 learned_addr learned my addr 172.21.15.114:0/219568497 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf82248700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 0x7fbf7c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 -- 172.21.15.114:0/219568497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 msgr2=0x7fbf7c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 0x7fbf7c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 -- 172.21.15.114:0/219568497 >> v1:172.21.15.114:6789/0 conn(0x7fbf7c0b5850 legacy=0x7fbf7c074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 -- 172.21.15.114:0/219568497 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0b43b0 con 0x7fbf7c073f90 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf82248700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 0x7fbf7c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf81a47700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c073f90 0x7fbf7c06f5d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009840 tx=0x7fbf6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6191ad59bfd294b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf80a45700 1 -- 172.21.15.114:0/219568497 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011830 con 0x7fbf7c073f90 2023-11-24T06:28:15.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf80a45700 1 -- 172.21.15.114:0/219568497 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c011990 con 0x7fbf7c073f90 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.661+0000 7fbf80a45700 1 -- 172.21.15.114:0/219568497 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011d20 con 0x7fbf7c073f90 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 -- 172.21.15.114:0/219568497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c073f90 msgr2=0x7fbf7c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c073f90 0x7fbf7c06f5d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009840 tx=0x7fbf6c009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 -- 172.21.15.114:0/219568497 shutdown_connections 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c06e9b0 0x7fbf7c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 --2- 172.21.15.114:0/219568497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c073f90 0x7fbf7c06f5d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 -- 172.21.15.114:0/219568497 >> 172.21.15.114:0/219568497 conn(0x7fbf7c0d0b40 msgr2=0x7fbf7c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 -- 172.21.15.114:0/219568497 shutdown_connections 2023-11-24T06:28:15.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.662+0000 7fbf83cab700 1 -- 172.21.15.114:0/219568497 wait complete. 2023-11-24T06:28:15.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.663+0000 7fbf83cab700 1 Processor -- start 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.663+0000 7fbf83cab700 1 -- start start 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 0x7fbf7c1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf7c073f90 0x7fbf7c0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c0b5850 0x7fbf7c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0c5e30 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1f4a10 con 0x7fbf7c073f90 2023-11-24T06:28:15.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1f4730 con 0x7fbf7c0b5850 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 0x7fbf7c1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 0x7fbf7c1443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42906/0 (socket says 172.21.15.114:42906) 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 -- 172.21.15.114:0/255800292 learned_addr learned my addr 172.21.15.114:0/255800292 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 -- 172.21.15.114:0/255800292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c0b5850 msgr2=0x7fbf7c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf7c0b5850 0x7fbf7c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 -- 172.21.15.114:0/255800292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf7c073f90 msgr2=0x7fbf7c0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf7c073f90 0x7fbf7c0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c0094a0 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf81a47700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 0x7fbf7c1443c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0090b0 tx=0x7fbf6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0038a0 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c01f070 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c003c60 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c15e5a0 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.664+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c15eb60 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.665+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf7c0b40d0 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf6c01a430 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf6006b080 0x7fbf6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf6c021070 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf81246700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf6006b080 0x7fbf6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7fbf60070ee0 0x7fbf60073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf600739d0 con 0x7fbf60070ee0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf6c08b520 con 0x7fbf7c06e9b0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf82248700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7fbf60070ee0 0x7fbf60073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.666+0000 7fbf81246700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf6006b080 0x7fbf6006d540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c145af0 tx=0x7fbf78007ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.667+0000 7fbf82248700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7fbf60070ee0 0x7fbf60073300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.667+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== osd.1 v2:172.21.15.114:6800/2279805148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbf600739d0 con 0x7fbf60070ee0 2023-11-24T06:28:15.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.682+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 --> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbf7c0b3df0 con 0x7fbf60070ee0 2023-11-24T06:28:15.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.682+0000 7fbf727fc700 1 -- 172.21.15.114:0/255800292 <== osd.1 v2:172.21.15.114:6800/2279805148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbf740089f0 con 0x7fbf60070ee0 2023-11-24T06:28:15.690 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:15.691 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:15.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7fbf60070ee0 msgr2=0x7fbf60073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6800/2279805148,v1:172.21.15.114:6801/2279805148] conn(0x7fbf60070ee0 0x7fbf60073300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf6006b080 msgr2=0x7fbf6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbf6006b080 0x7fbf6006d540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c145af0 tx=0x7fbf78007ad0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 msgr2=0x7fbf7c1443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 --2- 172.21.15.114:0/255800292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf7c06e9b0 0x7fbf7c1443c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0090b0 tx=0x7fbf6c004750 comp rx=0 tx=0).stop 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf82248700 1 -- 172.21.15.114:0/255800292 reap_dead start 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 shutdown_connections 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 >> 172.21.15.114:0/255800292 conn(0x7fbf7c0d0b40 msgr2=0x7fbf7c06e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 shutdown_connections 2023-11-24T06:28:15.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.683+0000 7fbf83cab700 1 -- 172.21.15.114:0/255800292 wait complete. 2023-11-24T06:28:15.698 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:28:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.817+0000 7f8e99c3b700 1 Processor -- start 2023-11-24T06:28:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.817+0000 7f8e99c3b700 1 -- start start 2023-11-24T06:28:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e99c3b700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8e9415e080 0x7f8e940b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e99c3b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e940af4d0 0x7f8e940afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e99c3b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940ae430 con 0x7f8e9406c000 2023-11-24T06:28:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e99c3b700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e940af0b0 con 0x7f8e9415e080 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e99c3b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940b05a0 con 0x7f8e940af4d0 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8e9415e080 0x7f8e940b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8e9415e080 0x7f8e940b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42916/0 (socket says 172.21.15.114:42916) 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 -- 172.21.15.114:0/3571982861 learned_addr learned my addr 172.21.15.114:0/3571982861 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 -- 172.21.15.114:0/3571982861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e940af4d0 msgr2=0x7f8e940afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 --2- 172.21.15.114:0/3571982861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e940af4d0 0x7f8e940afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 -- 172.21.15.114:0/3571982861 >> v1:172.21.15.172:6789/0 conn(0x7f8e9406c000 legacy=0x7f8e94073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.823 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 -- 172.21.15.114:0/3571982861 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940ae710 con 0x7f8e9415e080 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.818+0000 7f8e937fe700 1 --2- 172.21.15.114:0/3571982861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e9415e080 0x7f8e940b5050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009920 tx=0x7f8e88009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca349da65fe52588 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e927fc700 1 -- 172.21.15.114:0/3571982861 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88004230 con 0x7f8e9415e080 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e927fc700 1 -- 172.21.15.114:0/3571982861 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e88004390 con 0x7f8e9415e080 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e927fc700 1 -- 172.21.15.114:0/3571982861 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88009e70 con 0x7f8e9415e080 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 -- 172.21.15.114:0/3571982861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e9415e080 msgr2=0x7f8e940b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/3571982861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e9415e080 0x7f8e940b5050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009920 tx=0x7f8e88009c30 comp rx=0 tx=0).stop 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 -- 172.21.15.114:0/3571982861 shutdown_connections 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/3571982861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e9415e080 0x7f8e940b5050 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.824 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/3571982861 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e940af4d0 0x7f8e940afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 -- 172.21.15.114:0/3571982861 >> 172.21.15.114:0/3571982861 conn(0x7f8e94071b90 msgr2=0x7f8e94071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 -- 172.21.15.114:0/3571982861 shutdown_connections 2023-11-24T06:28:15.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.819+0000 7f8e99c3b700 1 -- 172.21.15.114:0/3571982861 wait complete. 2023-11-24T06:28:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.821+0000 7f8e99c3b700 1 Processor -- start 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.821+0000 7f8e99c3b700 1 -- start start 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e9406c000 0x7f8e9413e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9415e080 0x7f8e94141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94153d60 con 0x7f8e940af4d0 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94154040 con 0x7f8e9415e080 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e99c3b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94160f20 con 0x7f8e9406c000 2023-11-24T06:28:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42922/0 (socket says 172.21.15.114:42922) 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 -- 172.21.15.114:0/449287488 learned_addr learned my addr 172.21.15.114:0/449287488 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 -- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e9406c000 msgr2=0x7f8e9413e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e9406c000 0x7f8e9413e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 -- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9415e080 msgr2=0x7f8e94141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e93fff700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9415e080 0x7f8e94141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9415e080 0x7f8e94141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 -- 172.21.15.114:0/449287488 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e88009550 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.822+0000 7f8e92ffd700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400b500 tx=0x7f8e8400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84011830 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e840119b0 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84011d60 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e94161200 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9413eef0 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.823+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e94156940 con 0x7f8e940af4d0 2023-11-24T06:28:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.824+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8e8401a400 con 0x7f8e940af4d0 2023-11-24T06:28:15.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.824+0000 7f8e7bfff700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 0x7f8e8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.824+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e840896c0 con 0x7f8e940af4d0 2023-11-24T06:28:15.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.824+0000 7f8e937fe700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 0x7f8e8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.825+0000 7f8e937fe700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 0x7f8e8006d540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000c00 tx=0x7f8e88010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:15.829+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e840547f0 con 0x7f8e940af4d0 2023-11-24T06:28:16.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.018+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e940aecd0 con 0x7f8e8006b080 2023-11-24T06:28:16.021 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.019+0000 7f8e7bfff700 1 -- 172.21.15.114:0/449287488 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e88009e70 con 0x7f8e8006b080 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 msgr2=0x7f8e8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 0x7f8e8006d540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000c00 tx=0x7f8e88010000 comp rx=0 tx=0).stop 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 msgr2=0x7f8e941412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400b500 tx=0x7f8e8400b8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 shutdown_connections 2023-11-24T06:28:16.024 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f8e8006b080 0x7f8e8006d540 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e9415e080 0x7f8e94141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8e9406c000 0x7f8e9413e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 --2- 172.21.15.114:0/449287488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8e940af4d0 0x7f8e941412f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 >> 172.21.15.114:0/449287488 conn(0x7f8e94071b90 msgr2=0x7f8e9406f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 shutdown_connections 2023-11-24T06:28:16.025 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.021+0000 7f8e99c3b700 1 -- 172.21.15.114:0/449287488 wait complete. 2023-11-24T06:28:16.026 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:28:16.035 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:28:16.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:16.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.158+0000 7f3281e6e700 1 Processor -- start 2023-11-24T06:28:16.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.158+0000 7f3281e6e700 1 -- start start 2023-11-24T06:28:16.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f3281e6e700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f327c0746a0 0x7f327c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f3281e6e700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c0cf530 con 0x7f327c145620 2023-11-24T06:28:16.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f3281e6e700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c06b8f0 con 0x7f327c0cd810 2023-11-24T06:28:16.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f3281e6e700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327c06b700 con 0x7f327c0746a0 2023-11-24T06:28:16.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327b7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f327c145620 0x7f327c06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58636/0 (socket says 172.21.15.114:58636) 2023-11-24T06:28:16.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327b7fe700 1 -- 172.21.15.114:0/576115691 learned_addr learned my addr 172.21.15.114:0/576115691 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:16.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813533916 0 0) 0x7f327c0cf530 con 0x7f327c145620 2023-11-24T06:28:16.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327bfff700 1 --2- 172.21.15.114:0/576115691 >> v2:172.21.15.172:3300/0 conn(0x7f327c0746a0 0x7f327c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32640035b0 con 0x7f327c145620 2023-11-24T06:28:16.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2243549425 0 0) 0x7f32640035b0 con 0x7f327c145620 2023-11-24T06:28:16.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 >> v2:172.21.15.172:3300/0 conn(0x7f327c0746a0 msgr2=0x7f327c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 --2- 172.21.15.114:0/576115691 >> v2:172.21.15.172:3300/0 conn(0x7f327c0746a0 0x7f327c06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 >> v1:172.21.15.172:6789/0 conn(0x7f327c0cd810 legacy=0x7f327c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c06bbd0 con 0x7f327c145620 2023-11-24T06:28:16.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f326c001670 con 0x7f327c145620 2023-11-24T06:28:16.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 >> v1:172.21.15.114:6789/0 conn(0x7f327c145620 legacy=0x7f327c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3264003a00 0x7f3264003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f3264004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3264005210 0x7f326400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326400c080 con 0x7f3264004850 2023-11-24T06:28:16.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326400c1f0 con 0x7f3264003a00 2023-11-24T06:28:16.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.159+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326400c3b0 con 0x7f3264005210 2023-11-24T06:28:16.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f3264004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327b7fe700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3264003a00 0x7f3264003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327bfff700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3264005210 0x7f326400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 -- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3264005210 msgr2=0x7f326400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3264005210 0x7f326400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 -- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3264003a00 msgr2=0x7f3264003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3264003a00 0x7f3264003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 -- 172.21.15.114:0/576115691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640035b0 con 0x7f3264004850 2023-11-24T06:28:16.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327bfff700 1 -- 172.21.15.114:0/576115691 reap_dead start 2023-11-24T06:28:16.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327affd700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f3264004cd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f327c0cdca0 tx=0x7f3264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127a225256de77f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700052d0 con 0x7f3264004850 2023-11-24T06:28:16.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3270005450 con 0x7f3264004850 2023-11-24T06:28:16.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.160+0000 7f327a7fc700 1 -- 172.21.15.114:0/576115691 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700038c0 con 0x7f3264004850 2023-11-24T06:28:16.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 -- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 msgr2=0x7f3264004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f3264004cd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f327c0cdca0 tx=0x7f3264005880 comp rx=0 tx=0).stop 2023-11-24T06:28:16.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 -- 172.21.15.114:0/576115691 shutdown_connections 2023-11-24T06:28:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 --2- 172.21.15.114:0/576115691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f3264004cd0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 -- 172.21.15.114:0/576115691 >> 172.21.15.114:0/576115691 conn(0x7f327c0d0b40 msgr2=0x7f327c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 -- 172.21.15.114:0/576115691 shutdown_connections 2023-11-24T06:28:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.161+0000 7f3281e6e700 1 -- 172.21.15.114:0/576115691 wait complete. 2023-11-24T06:28:16.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.162+0000 7f3281e6e700 1 Processor -- start 2023-11-24T06:28:16.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.162+0000 7f3281e6e700 1 -- start start 2023-11-24T06:28:16.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f327c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f327c06df90 0x7f327c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f327c15c6f0 0x7f327c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c13aea0 con 0x7f3264004850 2023-11-24T06:28:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c06e950 con 0x7f327c15c6f0 2023-11-24T06:28:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c13b560 con 0x7f327c06df90 2023-11-24T06:28:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f327c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f327c15b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42938/0 (socket says 172.21.15.114:42938) 2023-11-24T06:28:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 -- 172.21.15.114:0/3883635188 learned_addr learned my addr 172.21.15.114:0/3883635188 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327bfff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f327c15c6f0 0x7f327c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327affd700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f327c06df90 0x7f327c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 -- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f327c06df90 msgr2=0x7f327c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f327c06df90 0x7f327c06e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 -- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f327c15c6f0 msgr2=0x7f327c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f327c15c6f0 0x7f327c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c15d460 con 0x7f3264004850 2023-11-24T06:28:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327bfff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f327c15c6f0 0x7f327c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f327b7fe700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f327c15b510 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f326c007290 tx=0x7f326c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c011580 con 0x7f3264004850 2023-11-24T06:28:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0116e0 con 0x7f3264004850 2023-11-24T06:28:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c15d740 con 0x7f3264004850 2023-11-24T06:28:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.163+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c011a70 con 0x7f3264004850 2023-11-24T06:28:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.164+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c15dd00 con 0x7f3264004850 2023-11-24T06:28:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.164+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f327c144e30 con 0x7f3264004850 2023-11-24T06:28:16.190 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f326c011bd0 con 0x7f3264004850 2023-11-24T06:28:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f326806b080 0x7f326806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f326c089430 con 0x7f3264004850 2023-11-24T06:28:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f327affd700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f326806b080 0x7f326806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f3268070ee0 0x7f3268073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32680739d0 con 0x7f3268070ee0 2023-11-24T06:28:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f326c08bad0 con 0x7f3264004850 2023-11-24T06:28:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.165+0000 7f327bfff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f3268070ee0 0x7f3268073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.166+0000 7f327affd700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f326806b080 0x7f326806d540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3270005040 tx=0x7f3270009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.166+0000 7f327bfff700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f3268070ee0 0x7f3268073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.166+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== osd.2 v2:172.21.15.114:6817/2005258798 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f32680739d0 con 0x7f3268070ee0 2023-11-24T06:28:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.181+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 --> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f327c145a00 con 0x7f3268070ee0 2023-11-24T06:28:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.181+0000 7f3263fff700 1 -- 172.21.15.114:0/3883635188 <== osd.2 v2:172.21.15.114:6817/2005258798 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f327400f040 con 0x7f3268070ee0 2023-11-24T06:28:16.195 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:16.195 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:16.195 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f3268070ee0 msgr2=0x7f3268073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6817/2005258798,v1:172.21.15.114:6820/2005258798] conn(0x7f3268070ee0 0x7f3268073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f326806b080 msgr2=0x7f326806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f326806b080 0x7f326806d540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3270005040 tx=0x7f3270009030 comp rx=0 tx=0).stop 2023-11-24T06:28:16.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 msgr2=0x7f327c15b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 --2- 172.21.15.114:0/3883635188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3264004850 0x7f327c15b510 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f326c007290 tx=0x7f326c0075a0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.199 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f327bfff700 1 -- 172.21.15.114:0/3883635188 reap_dead start 2023-11-24T06:28:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 shutdown_connections 2023-11-24T06:28:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 >> 172.21.15.114:0/3883635188 conn(0x7f327c0d0b40 msgr2=0x7f327c144720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 shutdown_connections 2023-11-24T06:28:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.182+0000 7f3281e6e700 1 -- 172.21.15.114:0/3883635188 wait complete. 2023-11-24T06:28:16.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:28:16.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.319+0000 7fb40650a700 1 Processor -- start 2023-11-24T06:28:16.322 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.319+0000 7fb40650a700 1 -- start start 2023-11-24T06:28:16.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb4001446a0 0x7fb4000b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4000adf00 0x7fb40006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40006c540 0x7fb40006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb400144a80 con 0x7fb4000adf00 2023-11-24T06:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40006c9c0 con 0x7fb4001446a0 2023-11-24T06:28:16.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb40650a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000af950 con 0x7fb40006c540 2023-11-24T06:28:16.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb3ff7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4000adf00 0x7fb40006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb3ff7fe700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4000adf00 0x7fb40006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42948/0 (socket says 172.21.15.114:42948) 2023-11-24T06:28:16.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.320+0000 7fb3ff7fe700 1 -- 172.21.15.114:0/4168278678 learned_addr learned my addr 172.21.15.114:0/4168278678 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.327 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb404aa7700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40006c540 0x7fb40006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3fffff700 1 --2- 172.21.15.114:0/4168278678 >> v2:172.21.15.172:3300/0 conn(0x7fb4001446a0 0x7fb4000b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 -- 172.21.15.114:0/4168278678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40006c540 msgr2=0x7fb40006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40006c540 0x7fb40006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 -- 172.21.15.114:0/4168278678 >> v2:172.21.15.172:3300/0 conn(0x7fb4001446a0 msgr2=0x7fb4000b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/4168278678 >> v2:172.21.15.172:3300/0 conn(0x7fb4001446a0 0x7fb4000b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 -- 172.21.15.114:0/4168278678 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000afc30 con 0x7fb4000adf00 2023-11-24T06:28:16.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4000adf00 0x7fb40006c000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb4000729a0 tx=0x7fb3f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27728edc06388b5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3feffd700 1 -- 172.21.15.114:0/4168278678 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000b870 con 0x7fb4000adf00 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3feffd700 1 -- 172.21.15.114:0/4168278678 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f000b9d0 con 0x7fb4000adf00 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb3feffd700 1 -- 172.21.15.114:0/4168278678 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000bd60 con 0x7fb4000adf00 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb40650a700 1 -- 172.21.15.114:0/4168278678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4000adf00 msgr2=0x7fb40006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.321+0000 7fb40650a700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4000adf00 0x7fb40006c000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb4000729a0 tx=0x7fb3f0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 -- 172.21.15.114:0/4168278678 shutdown_connections 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40006c540 0x7fb40006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 --2- 172.21.15.114:0/4168278678 >> v2:172.21.15.172:3300/0 conn(0x7fb4001446a0 0x7fb4000b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 --2- 172.21.15.114:0/4168278678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4000adf00 0x7fb40006c000 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 -- 172.21.15.114:0/4168278678 >> 172.21.15.114:0/4168278678 conn(0x7fb400071b90 msgr2=0x7fb400071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 -- 172.21.15.114:0/4168278678 shutdown_connections 2023-11-24T06:28:16.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.322+0000 7fb40650a700 1 -- 172.21.15.114:0/4168278678 wait complete. 2023-11-24T06:28:16.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.323+0000 7fb40650a700 1 Processor -- start 2023-11-24T06:28:16.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.323+0000 7fb40650a700 1 -- start start 2023-11-24T06:28:16.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4000adf00 0x7fb400158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4001446a0 0x7fb4001586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40013b4e0 con 0x7fb40006c540 2023-11-24T06:28:16.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40013d990 con 0x7fb4001446a0 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015a4d0 con 0x7fb4000adf00 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42954/0 (socket says 172.21.15.114:42954) 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 -- 172.21.15.114:0/1674974885 learned_addr learned my addr 172.21.15.114:0/1674974885 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4000adf00 0x7fb400158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb404aa7700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4001446a0 0x7fb4001586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 -- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4000adf00 msgr2=0x7fb400158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4000adf00 0x7fb400158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 -- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4001446a0 msgr2=0x7fb4001586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4001446a0 0x7fb4001586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 -- 172.21.15.114:0/1674974885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0009550 con 0x7fb40006c540 2023-11-24T06:28:16.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fffff700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d800 tx=0x7fb3f400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40126b0 con 0x7fb40006c540 2023-11-24T06:28:16.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f4012810 con 0x7fb40006c540 2023-11-24T06:28:16.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb400156f20 con 0x7fb40006c540 2023-11-24T06:28:16.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4012bc0 con 0x7fb40006c540 2023-11-24T06:28:16.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.324+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40013d620 con 0x7fb40006c540 2023-11-24T06:28:16.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.325+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4000753f0 con 0x7fb40006c540 2023-11-24T06:28:16.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.326+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb3f4012d20 con 0x7fb40006c540 2023-11-24T06:28:16.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.326+0000 7fb3fcff9700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 0x7fb3ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.326+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3f4089670 con 0x7fb40006c540 2023-11-24T06:28:16.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.326+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 0x7fb3ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.327+0000 7fb3ff7fe700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 0x7fb3ec06d540 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb4000729a0 tx=0x7fb3f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.331+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3f4054830 con 0x7fb40006c540 2023-11-24T06:28:16.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.522+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4000b04d0 con 0x7fb3ec06b080 2023-11-24T06:28:16.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.523+0000 7fb3fcff9700 1 -- 172.21.15.114:0/1674974885 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb3f000b7a0 con 0x7fb3ec06b080 2023-11-24T06:28:16.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 msgr2=0x7fb3ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 0x7fb3ec06d540 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb4000729a0 tx=0x7fb3f0008030 comp rx=0 tx=0).stop 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 msgr2=0x7fb4001569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d800 tx=0x7fb3f400db10 comp rx=0 tx=0).stop 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 shutdown_connections 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fb3ec06b080 0x7fb3ec06d540 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4001446a0 0x7fb4001586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40006c540 0x7fb4001569e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 --2- 172.21.15.114:0/1674974885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4000adf00 0x7fb400158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 >> 172.21.15.114:0/1674974885 conn(0x7fb400071b90 msgr2=0x7fb4000b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 shutdown_connections 2023-11-24T06:28:16.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.525+0000 7fb40650a700 1 -- 172.21.15.114:0/1674974885 wait complete. 2023-11-24T06:28:16.529 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:28:16.539 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:28:16.540 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:16.663 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.661+0000 7f3efa4d9700 1 Processor -- start 2023-11-24T06:28:16.664 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.661+0000 7f3efa4d9700 1 -- start start 2023-11-24T06:28:16.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ef40cd930 0x7f3ef40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40af4b0 0x7f3ef40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ef40cf550 0x7f3ef40cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef4162100 con 0x7f3ef40cf550 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef41623e0 con 0x7f3ef40cd930 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3efa4d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41626c0 con 0x7f3ef40af4b0 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ef40cf550 0x7f3ef40cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3ef40cf550 0x7f3ef40cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42966/0 (socket says 172.21.15.114:42966) 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 -- 172.21.15.114:0/1846369661 learned_addr learned my addr 172.21.15.114:0/1846369661 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef3fff700 1 --2- 172.21.15.114:0/1846369661 >> v2:172.21.15.172:3300/0 conn(0x7f3ef40cd930 0x7f3ef40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef37fe700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40af4b0 0x7f3ef40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 -- 172.21.15.114:0/1846369661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40af4b0 msgr2=0x7f3ef40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40af4b0 0x7f3ef40cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 -- 172.21.15.114:0/1846369661 >> v2:172.21.15.172:3300/0 conn(0x7f3ef40cd930 msgr2=0x7f3ef40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1846369661 >> v2:172.21.15.172:3300/0 conn(0x7f3ef40cd930 0x7f3ef40cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 -- 172.21.15.114:0/1846369661 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef41629a0 con 0x7f3ef40cf550 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.662+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40cf550 0x7f3ef40cf9e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40b2ee0 tx=0x7f3eec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03f818b51a94b56 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3ef2ffd700 1 -- 172.21.15.114:0/1846369661 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00e070 con 0x7f3ef40cf550 2023-11-24T06:28:16.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3ef2ffd700 1 -- 172.21.15.114:0/1846369661 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00b9c0 con 0x7f3ef40cf550 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3ef2ffd700 1 -- 172.21.15.114:0/1846369661 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00bd70 con 0x7f3ef40cf550 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1846369661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40cf550 msgr2=0x7f3ef40cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40cf550 0x7f3ef40cf9e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40b2ee0 tx=0x7f3eec00b330 comp rx=0 tx=0).stop 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1846369661 shutdown_connections 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40cf550 0x7f3ef40cf9e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1846369661 >> v2:172.21.15.172:3300/0 conn(0x7f3ef40cd930 0x7f3ef40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1846369661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40af4b0 0x7f3ef40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1846369661 >> 172.21.15.114:0/1846369661 conn(0x7f3ef4072710 msgr2=0x7f3ef4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1846369661 shutdown_connections 2023-11-24T06:28:16.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.663+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1846369661 wait complete. 2023-11-24T06:28:16.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.664+0000 7f3efa4d9700 1 Processor -- start 2023-11-24T06:28:16.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 -- start start 2023-11-24T06:28:16.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 0x7f3ef40cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cd930 0x7f3ef40b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef40cf550 0x7f3ef40b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41610c0 con 0x7f3ef40af4b0 2023-11-24T06:28:16.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40cbfd0 con 0x7f3ef40cf550 2023-11-24T06:28:16.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3efa4d9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40c6620 con 0x7f3ef40cd930 2023-11-24T06:28:16.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3ef3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 0x7f3ef40cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3ef3fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 0x7f3ef40cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42968/0 (socket says 172.21.15.114:42968) 2023-11-24T06:28:16.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.665+0000 7f3ef3fff700 1 -- 172.21.15.114:0/1649632703 learned_addr learned my addr 172.21.15.114:0/1649632703 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef40cf550 0x7f3ef40b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef37fe700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cd930 0x7f3ef40b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 -- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cd930 msgr2=0x7f3ef40b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cd930 0x7f3ef40b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 -- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef40cf550 msgr2=0x7f3ef40b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef40cf550 0x7f3ef40b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec009b30 con 0x7f3ef40af4b0 2023-11-24T06:28:16.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef40cf550 0x7f3ef40b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:16.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef37fe700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cd930 0x7f3ef40b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:16.681 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef3fff700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 0x7f3ef40cba90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800d800 tx=0x7f3ee800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80125f0 con 0x7f3ef40af4b0 2023-11-24T06:28:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8012750 con 0x7f3ef40af4b0 2023-11-24T06:28:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4160790 con 0x7f3ef40af4b0 2023-11-24T06:28:16.682 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8012b00 con 0x7f3ef40af4b0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.666+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef4160d50 con 0x7f3ef40af4b0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.667+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ef40ccf30 con 0x7f3ef40af4b0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.667+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3ee8012c60 con 0x7f3ef40af4b0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef0ff9700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3ee006b080 0x7f3ee006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ee808a2e0 con 0x7f3ef40af4b0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef37fe700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3ee006b080 0x7f3ee006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef0ff9700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f3ee0070ee0 0x7f3ee0073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ee00739d0 con 0x7f3ee0070ee0 2023-11-24T06:28:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee808a730 con 0x7f3ef40af4b0 2023-11-24T06:28:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f3ee0070ee0 0x7f3ee0073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef37fe700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3ee006b080 0x7f3ee006d540 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af2f0 tx=0x7f3ee4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.668+0000 7f3ef8a76700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f3ee0070ee0 0x7f3ee0073300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.669+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== osd.3 v2:172.21.15.114:6816/783795072 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3ee00739d0 con 0x7f3ee0070ee0 2023-11-24T06:28:16.686 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 --> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3ef4161200 con 0x7f3ee0070ee0 2023-11-24T06:28:16.686 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3ef0ff9700 1 -- 172.21.15.114:0/1649632703 <== osd.3 v2:172.21.15.114:6816/783795072 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3eec00bb30 con 0x7f3ee0070ee0 2023-11-24T06:28:16.687 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:16.687 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:16.688 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f3ee0070ee0 msgr2=0x7f3ee0073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6816/783795072,v1:172.21.15.114:6818/783795072] conn(0x7f3ee0070ee0 0x7f3ee0073300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3ee006b080 msgr2=0x7f3ee006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f3ee006b080 0x7f3ee006d540 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af2f0 tx=0x7f3ee4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 msgr2=0x7f3ef40cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 --2- 172.21.15.114:0/1649632703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ef40af4b0 0x7f3ef40cba90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800d800 tx=0x7f3ee800dbc0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3ef8a76700 1 -- 172.21.15.114:0/1649632703 reap_dead start 2023-11-24T06:28:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 shutdown_connections 2023-11-24T06:28:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 >> 172.21.15.114:0/1649632703 conn(0x7f3ef4072710 msgr2=0x7f3ef406df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 shutdown_connections 2023-11-24T06:28:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.684+0000 7f3efa4d9700 1 -- 172.21.15.114:0/1649632703 wait complete. 2023-11-24T06:28:16.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:28:16.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:16.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.823+0000 7f2431c16700 1 Processor -- start 2023-11-24T06:28:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.823+0000 7f2431c16700 1 -- start start 2023-11-24T06:28:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f2431c16700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f242c0d2240 0x7f242c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f2431c16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f242c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f2431c16700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c0b1d40 con 0x7f242c15ddd0 2023-11-24T06:28:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f2431c16700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c0b28c0 con 0x7f242c0d2240 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f2431c16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c06c000 con 0x7f242c15e1b0 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242b7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f242c15ddd0 0x7f242c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58686/0 (socket says 172.21.15.114:58686) 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242b7fe700 1 -- 172.21.15.114:0/496306844 learned_addr learned my addr 172.21.15.114:0/496306844 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242affd700 1 --2- 172.21.15.114:0/496306844 >> v2:172.21.15.172:3300/0 conn(0x7f242c0d2240 0x7f242c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242bfff700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f242c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755775599 0 0) 0x7f242c0b1d40 con 0x7f242c15ddd0 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24180035b0 con 0x7f242c15ddd0 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285116147 0 0) 0x7f24180035b0 con 0x7f242c15ddd0 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 msgr2=0x7f242c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f242c0cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 >> v2:172.21.15.172:3300/0 conn(0x7f242c0d2240 msgr2=0x7f242c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 --2- 172.21.15.114:0/496306844 >> v2:172.21.15.172:3300/0 conn(0x7f242c0d2240 0x7f242c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0b1f90 con 0x7f242c15ddd0 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f24240019f0 con 0x7f242c15ddd0 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242affd700 1 --2- 172.21.15.114:0/496306844 >> v2:172.21.15.172:3300/0 conn(0x7f242c0d2240 0x7f242c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 >> v1:172.21.15.114:6789/0 conn(0x7f242c15ddd0 legacy=0x7f242c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f2418003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418003f80 0x7f2418004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.824+0000 7f242a7fc700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f2418005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0b28c0 con 0x7f2418004d90 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c06c000 con 0x7f2418003f80 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241800baf0 con 0x7f242c15e1b0 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f2418005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242affd700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418003f80 0x7f2418004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242b7fe700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f2418003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 -- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 msgr2=0x7f2418003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c15e1b0 0x7f2418003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 -- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418003f80 msgr2=0x7f2418004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418003f80 0x7f2418004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 -- 172.21.15.114:0/496306844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180035b0 con 0x7f2418004d90 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242bfff700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f2418005210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f241c009b60 tx=0x7f2418005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e088a9b6d9fe81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00e6c0 con 0x7f2418004d90 2023-11-24T06:28:16.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00e820 con 0x7f2418004d90 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.825+0000 7f242a7fc700 1 -- 172.21.15.114:0/496306844 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00eb90 con 0x7f2418004d90 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 -- 172.21.15.114:0/496306844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 msgr2=0x7f2418005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 --2- 172.21.15.114:0/496306844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f2418005210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f241c009b60 tx=0x7f2418005880 comp rx=0 tx=0).stop 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f242bfff700 1 -- 172.21.15.114:0/496306844 reap_dead start 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 -- 172.21.15.114:0/496306844 shutdown_connections 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 -- 172.21.15.114:0/496306844 >> 172.21.15.114:0/496306844 conn(0x7f242c0731a0 msgr2=0x7f242c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 -- 172.21.15.114:0/496306844 shutdown_connections 2023-11-24T06:28:16.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.826+0000 7f2431c16700 1 -- 172.21.15.114:0/496306844 wait complete. 2023-11-24T06:28:16.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.827+0000 7f2431c16700 1 Processor -- start 2023-11-24T06:28:16.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.827+0000 7f2431c16700 1 -- start start 2023-11-24T06:28:16.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f242c1f6f30 0x7f242c1f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c0b0450 0x7f242c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0b0e20 con 0x7f2418004d90 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1f5080 con 0x7f242c1f6f30 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f2431c16700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1f4a50 con 0x7f242c0b0450 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42990/0 (socket says 172.21.15.114:42990) 2023-11-24T06:28:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 -- 172.21.15.114:0/1122696396 learned_addr learned my addr 172.21.15.114:0/1122696396 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242bfff700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c0b0450 0x7f242c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242affd700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f242c1f6f30 0x7f242c1f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 -- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c0b0450 msgr2=0x7f242c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c0b0450 0x7f242c0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 -- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f242c1f6f30 msgr2=0x7f242c1f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f242c1f6f30 0x7f242c1f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 -- 172.21.15.114:0/1122696396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0b2ed0 con 0x7f2418004d90 2023-11-24T06:28:16.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.828+0000 7f242b7fe700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f242400a570 tx=0x7f242400a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240133f0 con 0x7f2418004d90 2023-11-24T06:28:16.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424013550 con 0x7f2418004d90 2023-11-24T06:28:16.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424013940 con 0x7f2418004d90 2023-11-24T06:28:16.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c1f79f0 con 0x7f2418004d90 2023-11-24T06:28:16.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c1f4d10 con 0x7f2418004d90 2023-11-24T06:28:16.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.829+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c1f4ed0 con 0x7f2418004d90 2023-11-24T06:28:16.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.830+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2424013aa0 con 0x7f2418004d90 2023-11-24T06:28:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.830+0000 7f2423fff700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 0x7f240c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.830+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f242408a4c0 con 0x7f2418004d90 2023-11-24T06:28:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.830+0000 7f242affd700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 0x7f240c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.831+0000 7f242affd700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 0x7f240c06d540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2414009470 tx=0x7f2414009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.845 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:16.835+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24240555f0 con 0x7f2418004d90 2023-11-24T06:28:17.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.028+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f242c0b2d50 con 0x7f240c06b080 2023-11-24T06:28:17.031 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.028+0000 7f2423fff700 1 -- 172.21.15.114:0/1122696396 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2414004030 con 0x7f240c06b080 2023-11-24T06:28:17.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 msgr2=0x7f240c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 0x7f240c06d540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2414009470 tx=0x7f2414009210 comp rx=0 tx=0).stop 2023-11-24T06:28:17.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 msgr2=0x7f242c1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f242400a570 tx=0x7f242400a880 comp rx=0 tx=0).stop 2023-11-24T06:28:17.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 shutdown_connections 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f240c06b080 0x7f240c06d540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f242c0b0450 0x7f242c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2418004d90 0x7f242c1582e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 --2- 172.21.15.114:0/1122696396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f242c1f6f30 0x7f242c1f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 >> 172.21.15.114:0/1122696396 conn(0x7f242c0731a0 msgr2=0x7f242c09aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.034 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 shutdown_connections 2023-11-24T06:28:17.035 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.030+0000 7f2431c16700 1 -- 172.21.15.114:0/1122696396 wait complete. 2023-11-24T06:28:17.035 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:28:17.044 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:28:17.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:17.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.166+0000 7fe56c905700 1 Processor -- start 2023-11-24T06:28:17.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.166+0000 7fe56c905700 1 -- start start 2023-11-24T06:28:17.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe56c905700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe56806dad0 0x7fe568166390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe56c905700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680b3940 con 0x7fe5680b3dd0 2023-11-24T06:28:17.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe56c905700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680cf5f0 con 0x7fe5680cf110 2023-11-24T06:28:17.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe56c905700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680cf8d0 con 0x7fe56806dad0 2023-11-24T06:28:17.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe565d9b700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe5680b3dd0 0x7fe56815def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58708/0 (socket says 172.21.15.114:58708) 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.167+0000 7fe565d9b700 1 -- 172.21.15.114:0/238732487 learned_addr learned my addr 172.21.15.114:0/238732487 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe566d9d700 1 --2- 172.21.15.114:0/238732487 >> v2:172.21.15.172:3300/0 conn(0x7fe56806dad0 0x7fe568166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097878357 0 0) 0x7fe5680b3940 con 0x7fe5680b3dd0 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5500035b0 con 0x7fe5680b3dd0 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145976014 0 0) 0x7fe5500035b0 con 0x7fe5680b3dd0 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 >> v2:172.21.15.172:3300/0 conn(0x7fe56806dad0 msgr2=0x7fe568166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 --2- 172.21.15.114:0/238732487 >> v2:172.21.15.172:3300/0 conn(0x7fe56806dad0 0x7fe568166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 >> v1:172.21.15.172:6789/0 conn(0x7fe5680cf110 legacy=0x7fe5680714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680b32f0 con 0x7fe5680b3dd0 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7fe558001670 con 0x7fe5680b3dd0 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 >> v1:172.21.15.114:6789/0 conn(0x7fe5680b3dd0 legacy=0x7fe56815def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe550003a00 0x7fe550003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe550004400 0x7fe5500048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe55000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680cf8d0 con 0x7fe5500050d0 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55000bf80 con 0x7fe550003a00 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55000c140 con 0x7fe550004400 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe566d9d700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe55000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe56659c700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe550003a00 0x7fe550003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.168+0000 7fe565d9b700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe550004400 0x7fe5500048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 -- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe550004400 msgr2=0x7fe5500048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe550004400 0x7fe5500048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 -- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe550003a00 msgr2=0x7fe550003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe550003a00 0x7fe550003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 -- 172.21.15.114:0/238732487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5500035b0 con 0x7fe5500050d0 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 -- 172.21.15.114:0/238732487 reap_dead start 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe566d9d700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe55000ba10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5680d0a60 tx=0x7fe550005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f85ee7325da6ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560003c20 con 0x7fe5500050d0 2023-11-24T06:28:17.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56000f040 con 0x7fe5500050d0 2023-11-24T06:28:17.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56559a700 1 -- 172.21.15.114:0/238732487 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000e9d0 con 0x7fe5500050d0 2023-11-24T06:28:17.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 -- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 msgr2=0x7fe55000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe55000ba10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5680d0a60 tx=0x7fe550005880 comp rx=0 tx=0).stop 2023-11-24T06:28:17.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 -- 172.21.15.114:0/238732487 shutdown_connections 2023-11-24T06:28:17.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 --2- 172.21.15.114:0/238732487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe55000ba10 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 -- 172.21.15.114:0/238732487 >> 172.21.15.114:0/238732487 conn(0x7fe5680b1740 msgr2=0x7fe5680b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.169+0000 7fe56c905700 1 -- 172.21.15.114:0/238732487 shutdown_connections 2023-11-24T06:28:17.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.170+0000 7fe56c905700 1 -- 172.21.15.114:0/238732487 wait complete. 2023-11-24T06:28:17.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.171+0000 7fe56c905700 1 Processor -- start 2023-11-24T06:28:17.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.171+0000 7fe56c905700 1 -- start start 2023-11-24T06:28:17.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.171+0000 7fe56c905700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe56814a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe56814bc60 0x7fe5681f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5681f3370 0x7fe5681f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56814a850 con 0x7fe5500050d0 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56814c4d0 con 0x7fe56814bc60 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56814c7b0 con 0x7fe5681f3370 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe56814a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe56814a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43010/0 (socket says 172.21.15.114:43010) 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 -- 172.21.15.114:0/385649569 learned_addr learned my addr 172.21.15.114:0/385649569 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe565d9b700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe56814bc60 0x7fe5681f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe566d9d700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5681f3370 0x7fe5681f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 -- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5681f3370 msgr2=0x7fe5681f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5681f3370 0x7fe5681f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 -- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe56814bc60 msgr2=0x7fe5681f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.184 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe56814bc60 0x7fe5681f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56814ca90 con 0x7fe5500050d0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe566d9d700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5681f3370 0x7fe5681f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56659c700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe56814a310 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00f840 tx=0x7fe55c00fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c005320 con 0x7fe5500050d0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c0054a0 con 0x7fe5500050d0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56814cd70 con 0x7fe5500050d0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c006aa0 con 0x7fe5500050d0 2023-11-24T06:28:17.185 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.172+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56814c0e0 con 0x7fe5500050d0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.173+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5680b41f0 con 0x7fe5500050d0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe55c003680 con 0x7fe5500050d0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806b080 0x7fe54806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe55c0893f0 con 0x7fe5500050d0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe565d9b700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806b080 0x7fe54806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7fe548070ee0 0x7fe548073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5480739d0 con 0x7fe548070ee0 2023-11-24T06:28:17.186 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe55c089840 con 0x7fe5500050d0 2023-11-24T06:28:17.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe566d9d700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7fe548070ee0 0x7fe548073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.174+0000 7fe565d9b700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806b080 0x7fe54806d540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe56815d5d0 tx=0x7fe558001b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.175+0000 7fe566d9d700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7fe548070ee0 0x7fe548073300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.187 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.176+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== osd.4 v2:172.21.15.172:6800/517626883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe5480739d0 con 0x7fe548070ee0 2023-11-24T06:28:17.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 --> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe568068b00 con 0x7fe548070ee0 2023-11-24T06:28:17.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe556ffd700 1 -- 172.21.15.114:0/385649569 <== osd.4 v2:172.21.15.172:6800/517626883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe568068b00 con 0x7fe548070ee0 2023-11-24T06:28:17.193 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:17.194 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:17.194 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7fe548070ee0 msgr2=0x7fe548073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.172:6800/517626883,v1:172.21.15.172:6801/517626883] conn(0x7fe548070ee0 0x7fe548073300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806b080 msgr2=0x7fe54806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fe54806b080 0x7fe54806d540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe56815d5d0 tx=0x7fe558001b90 comp rx=0 tx=0).stop 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 msgr2=0x7fe56814a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 --2- 172.21.15.114:0/385649569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5500050d0 0x7fe56814a310 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00f840 tx=0x7fe55c00fb50 comp rx=0 tx=0).stop 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe566d9d700 1 -- 172.21.15.114:0/385649569 reap_dead start 2023-11-24T06:28:17.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 shutdown_connections 2023-11-24T06:28:17.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 >> 172.21.15.114:0/385649569 conn(0x7fe5680b1740 msgr2=0x7fe5680cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 shutdown_connections 2023-11-24T06:28:17.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.191+0000 7fe56c905700 1 -- 172.21.15.114:0/385649569 wait complete. 2023-11-24T06:28:17.207 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:28:17.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.326+0000 7f5343a24700 1 Processor -- start 2023-11-24T06:28:17.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.326+0000 7f5343a24700 1 -- start start 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5343a24700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f533c15e080 0x7f533c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5343a24700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f533c06c710 0x7f533c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5343a24700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0ae430 con 0x7f533c0af4d0 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5343a24700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c0af0b0 con 0x7f533c15e080 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5343a24700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c0b05a0 con 0x7f533c06c710 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f533c15e080 0x7f533c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f533c15e080 0x7f533c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43020/0 (socket says 172.21.15.114:43020) 2023-11-24T06:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 -- 172.21.15.114:0/2999010580 learned_addr learned my addr 172.21.15.114:0/2999010580 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5341fc1700 1 --2- 172.21.15.114:0/2999010580 >> v2:172.21.15.172:3300/0 conn(0x7f533c06c710 0x7f533c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 -- 172.21.15.114:0/2999010580 >> v2:172.21.15.172:3300/0 conn(0x7f533c06c710 msgr2=0x7f533c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 --2- 172.21.15.114:0/2999010580 >> v2:172.21.15.172:3300/0 conn(0x7f533c06c710 0x7f533c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 -- 172.21.15.114:0/2999010580 >> v1:172.21.15.172:6789/0 conn(0x7f533c0af4d0 legacy=0x7f533c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 -- 172.21.15.114:0/2999010580 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0ae710 con 0x7f533c15e080 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f53417c0700 1 --2- 172.21.15.114:0/2999010580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c0b5050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f533c06e060 tx=0x7f5338009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa8fc69c005b45e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.332 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.327+0000 7f5333fff700 1 -- 172.21.15.114:0/2999010580 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800b860 con 0x7f533c15e080 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5333fff700 1 -- 172.21.15.114:0/2999010580 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533800b9c0 con 0x7f533c15e080 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5333fff700 1 -- 172.21.15.114:0/2999010580 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800bd50 con 0x7f533c15e080 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 -- 172.21.15.114:0/2999010580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 msgr2=0x7f533c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 --2- 172.21.15.114:0/2999010580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c0b5050 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f533c06e060 tx=0x7f5338009c00 comp rx=0 tx=0).stop 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 -- 172.21.15.114:0/2999010580 shutdown_connections 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 --2- 172.21.15.114:0/2999010580 >> v2:172.21.15.172:3300/0 conn(0x7f533c06c710 0x7f533c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 --2- 172.21.15.114:0/2999010580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c0b5050 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.333 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 -- 172.21.15.114:0/2999010580 >> 172.21.15.114:0/2999010580 conn(0x7f533c071b90 msgr2=0x7f533c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 -- 172.21.15.114:0/2999010580 shutdown_connections 2023-11-24T06:28:17.334 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.328+0000 7f5343a24700 1 -- 172.21.15.114:0/2999010580 wait complete. 2023-11-24T06:28:17.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.329+0000 7f5343a24700 1 Processor -- start 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.329+0000 7f5343a24700 1 -- start start 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f533c06c710 0x7f533c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f533c0af4d0 0x7f533c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1538d0 con 0x7f533c15e080 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c157dc0 con 0x7f533c06c710 2023-11-24T06:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5343a24700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1580a0 con 0x7f533c0af4d0 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43028/0 (socket says 172.21.15.114:43028) 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 -- 172.21.15.114:0/2219599462 learned_addr learned my addr 172.21.15.114:0/2219599462 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 -- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f533c0af4d0 msgr2=0x7f533c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f53417c0700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f533c06c710 0x7f533c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5340fbf700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f533c0af4d0 0x7f533c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f533c0af4d0 0x7f533c153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.330+0000 7f5341fc1700 1 -- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f533c06c710 msgr2=0x7f533c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5341fc1700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f533c06c710 0x7f533c157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5341fc1700 1 -- 172.21.15.114:0/2219599462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338009550 con 0x7f533c15e080 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5341fc1700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5334007e30 tx=0x7f533400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400eea0 con 0x7f533c15e080 2023-11-24T06:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334014420 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c158380 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53340147d0 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.331+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c158940 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.332+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c0753f0 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.332+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5334014930 con 0x7f533c15e080 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.332+0000 7f5331ffb700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 0x7f532006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.340 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.333+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5334089cc0 con 0x7f533c15e080 2023-11-24T06:28:17.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.333+0000 7f53417c0700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 0x7f532006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.333+0000 7f53417c0700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 0x7f532006d540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5338009c40 tx=0x7f5338008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.341 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.337+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5334054e80 con 0x7f533c15e080 2023-11-24T06:28:17.529 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.526+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f533c0aecd0 con 0x7f532006b080 2023-11-24T06:28:17.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.526+0000 7f5331ffb700 1 -- 172.21.15.114:0/2219599462 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f533801f070 con 0x7f532006b080 2023-11-24T06:28:17.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 msgr2=0x7f532006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 0x7f532006d540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5338009c40 tx=0x7f5338008030 comp rx=0 tx=0).stop 2023-11-24T06:28:17.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 msgr2=0x7f533c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5334007e30 tx=0x7f533400e3c0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 shutdown_connections 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f532006b080 0x7f532006d540 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15e080 0x7f533c154330 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f533c06c710 0x7f533c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 --2- 172.21.15.114:0/2219599462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f533c0af4d0 0x7f533c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 >> 172.21.15.114:0/2219599462 conn(0x7f533c071b90 msgr2=0x7f533c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 shutdown_connections 2023-11-24T06:28:17.533 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.528+0000 7f5343a24700 1 -- 172.21.15.114:0/2219599462 wait complete. 2023-11-24T06:28:17.534 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:28:17.544 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:28:17.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:17.667 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.664+0000 7f118d56d700 1 Processor -- start 2023-11-24T06:28:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.664+0000 7f118d56d700 1 -- start start 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f118d56d700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f118815dfe0 0x7f11880d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f118d56d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f11880b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f118d56d700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11880d1e60 con 0x7f11880d39a0 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f118d56d700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880d2ae0 con 0x7f118815dfe0 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f118d56d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880d2140 con 0x7f11880b6040 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f11867fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f11880d39a0 0x7f11880b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58744/0 (socket says 172.21.15.114:58744) 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f11867fc700 1 -- 172.21.15.114:0/549188812 learned_addr learned my addr 172.21.15.114:0/549188812 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1186ffd700 1 --2- 172.21.15.114:0/549188812 >> v2:172.21.15.172:3300/0 conn(0x7f118815dfe0 0x7f11880d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f11877fe700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f11880b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187743220 0 0) 0x7f11880d1e60 con 0x7f11880d39a0 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11700035b0 con 0x7f11880d39a0 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335154469 0 0) 0x7f11700035b0 con 0x7f11880d39a0 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 msgr2=0x7f11880b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f11880b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 >> v2:172.21.15.172:3300/0 conn(0x7f118815dfe0 msgr2=0x7f11880d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 --2- 172.21.15.114:0/549188812 >> v2:172.21.15.172:3300/0 conn(0x7f118815dfe0 0x7f11880d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.674 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.665+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880d2420 con 0x7f11880d39a0 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f1178001670 con 0x7f11880d39a0 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 >> v1:172.21.15.114:6789/0 conn(0x7f11880d39a0 legacy=0x7f11880b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1170003a00 0x7f1170003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f1170004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1170005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880d2ae0 con 0x7f1170004d90 2023-11-24T06:28:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880d2140 con 0x7f1170003a00 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117000ba10 con 0x7f11880b6040 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1170005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11867fc700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f1170004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1186ffd700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1170003a00 0x7f1170003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 -- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 msgr2=0x7f1170004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11880b6040 0x7f1170004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 -- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1170003a00 msgr2=0x7f1170003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1170003a00 0x7f1170003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 -- 172.21.15.114:0/549188812 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700035b0 con 0x7f1170004d90 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f11877fe700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1170005210 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fc0 tx=0x7f1170005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c81ac1678b2497 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000ebd0 con 0x7f1170004d90 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000f040 con 0x7f1170004d90 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.666+0000 7f1185ffb700 1 -- 172.21.15.114:0/549188812 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180014480 con 0x7f1170004d90 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 -- 172.21.15.114:0/549188812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 msgr2=0x7f1170005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 --2- 172.21.15.114:0/549188812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1170005210 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fc0 tx=0x7f1170005880 comp rx=0 tx=0).stop 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f11877fe700 1 -- 172.21.15.114:0/549188812 reap_dead start 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 -- 172.21.15.114:0/549188812 shutdown_connections 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 -- 172.21.15.114:0/549188812 >> 172.21.15.114:0/549188812 conn(0x7f11880d0b10 msgr2=0x7f11880d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 -- 172.21.15.114:0/549188812 shutdown_connections 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.667+0000 7f118d56d700 1 -- 172.21.15.114:0/549188812 wait complete. 2023-11-24T06:28:17.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.668+0000 7f118d56d700 1 Processor -- start 2023-11-24T06:28:17.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.668+0000 7f118d56d700 1 -- start start 2023-11-24T06:28:17.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1188156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188154160 0x7f1188159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.683 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f118815a520 0x7f11881548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118815a9a0 con 0x7f1170004d90 2023-11-24T06:28:17.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11881545e0 con 0x7f1188154160 2023-11-24T06:28:17.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f118d56d700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1188155680 con 0x7f118815a520 2023-11-24T06:28:17.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1188156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.684 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1188156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43046/0 (socket says 172.21.15.114:43046) 2023-11-24T06:28:17.685 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 -- 172.21.15.114:0/3133481278 learned_addr learned my addr 172.21.15.114:0/3133481278 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:17.685 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f11877fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f118815a520 0x7f11881548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.685 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f11867fc700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188154160 0x7f1188159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 -- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f118815a520 msgr2=0x7f11881548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f118815a520 0x7f11881548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.687 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 -- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188154160 msgr2=0x7f1188159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188154160 0x7f1188159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188155960 con 0x7f1170004d90 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f11877fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f118815a520 0x7f11881548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.669+0000 7f1186ffd700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1188156720 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f117c011aa0 tx=0x7f117c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00f980 con 0x7f1170004d90 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00fb00 con 0x7f1170004d90 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00feb0 con 0x7f1170004d90 2023-11-24T06:28:17.688 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1188155c40 con 0x7f1170004d90 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1188156200 con 0x7f1170004d90 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.670+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f11880d2f00 con 0x7f1170004d90 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.671+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f117c014440 con 0x7f1170004d90 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.671+0000 7f116f7fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f117406b080 0x7f117406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f117c089db0 con 0x7f1170004d90 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f11867fc700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f117406b080 0x7f117406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f116f7fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7f1174070ee0 0x7f1174073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11740739d0 con 0x7f1174070ee0 2023-11-24T06:28:17.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117c08c450 con 0x7f1170004d90 2023-11-24T06:28:17.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f11867fc700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f117406b080 0x7f117406d540 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f11877fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7f1174070ee0 0x7f1174073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.672+0000 7f11877fe700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7f1174070ee0 0x7f1174073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.690 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.673+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== osd.5 v2:172.21.15.172:6816/1185079648 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f11740739d0 con 0x7f1174070ee0 2023-11-24T06:28:17.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.688+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 --> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f11880d5000 con 0x7f1174070ee0 2023-11-24T06:28:17.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.688+0000 7f116f7fe700 1 -- 172.21.15.114:0/3133481278 <== osd.5 v2:172.21.15.172:6816/1185079648 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f11880d5000 con 0x7f1174070ee0 2023-11-24T06:28:17.692 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:17.692 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:17.692 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.688+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7f1174070ee0 msgr2=0x7f1174073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.688+0000 7f118d56d700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.172:6816/1185079648,v1:172.21.15.172:6817/1185079648] conn(0x7f1174070ee0 0x7f1174073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f117406b080 msgr2=0x7f117406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f117406b080 0x7f117406d540 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178009040 comp rx=0 tx=0).stop 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 msgr2=0x7f1188156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 --2- 172.21.15.114:0/3133481278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1170004d90 0x7f1188156720 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f117c011aa0 tx=0x7f117c011e60 comp rx=0 tx=0).stop 2023-11-24T06:28:17.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f11877fe700 1 -- 172.21.15.114:0/3133481278 reap_dead start 2023-11-24T06:28:17.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 shutdown_connections 2023-11-24T06:28:17.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 >> 172.21.15.114:0/3133481278 conn(0x7f11880d0b10 msgr2=0x7f11880b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 shutdown_connections 2023-11-24T06:28:17.694 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.689+0000 7f118d56d700 1 -- 172.21.15.114:0/3133481278 wait complete. 2023-11-24T06:28:17.704 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:28:17.825 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.822+0000 7f51d17db700 1 Processor -- start 2023-11-24T06:28:17.826 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.822+0000 7f51d17db700 1 -- start start 2023-11-24T06:28:17.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51d17db700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f51cc15e0f0 0x7f51cc0b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51d17db700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc06bc00 con 0x7f51cc0b2b60 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51d17db700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc15e4d0 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51d17db700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51cc06aff0 con 0x7f51cc15e0f0 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f51cc15e0f0 0x7f51cc0b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51ca7fc700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f51cc0b2b60 0x7f51cc0ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58764/0 (socket says 172.21.15.114:58764) 2023-11-24T06:28:17.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f51cc15e0f0 0x7f51cc0b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43056/0 (socket says 172.21.15.114:43056) 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51ca7fc700 1 -- 172.21.15.114:0/2283837359 learned_addr learned my addr 172.21.15.114:0/2283837359 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 -- 172.21.15.114:0/2283837359 >> v1:172.21.15.172:6789/0 conn(0x7f51cc0cf2a0 legacy=0x7f51cc0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 -- 172.21.15.114:0/2283837359 >> v1:172.21.15.114:6789/0 conn(0x7f51cc0b2b60 legacy=0x7f51cc0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 -- 172.21.15.114:0/2283837359 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc06b2d0 con 0x7f51cc15e0f0 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51c9ffb700 1 -- 172.21.15.114:0/2283837359 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807928598 0 0) 0x7f51cc06bc00 con 0x7f51cc0b2b60 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51caffd700 1 --2- 172.21.15.114:0/2283837359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0b9700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009840 tx=0x7f51bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e5f3c90a7344f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51c9ffb700 1 -- 172.21.15.114:0/2283837359 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc004230 con 0x7f51cc15e0f0 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51c9ffb700 1 -- 172.21.15.114:0/2283837359 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc004390 con 0x7f51cc15e0f0 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.823+0000 7f51c9ffb700 1 -- 172.21.15.114:0/2283837359 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0036a0 con 0x7f51cc15e0f0 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 -- 172.21.15.114:0/2283837359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc15e0f0 msgr2=0x7f51cc0b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 --2- 172.21.15.114:0/2283837359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0b9700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009840 tx=0x7f51bc009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 -- 172.21.15.114:0/2283837359 shutdown_connections 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 --2- 172.21.15.114:0/2283837359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0b9700 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 -- 172.21.15.114:0/2283837359 >> 172.21.15.114:0/2283837359 conn(0x7f51cc0b5ce0 msgr2=0x7f51cc0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 -- 172.21.15.114:0/2283837359 shutdown_connections 2023-11-24T06:28:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.824+0000 7f51d17db700 1 -- 172.21.15.114:0/2283837359 wait complete. 2023-11-24T06:28:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.825+0000 7f51d17db700 1 Processor -- start 2023-11-24T06:28:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.825+0000 7f51d17db700 1 -- start start 2023-11-24T06:28:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51cc0b2b60 0x7f51cc160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc160a10 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc15fe00 con 0x7f51cc15e0f0 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51d17db700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc1600e0 con 0x7f51cc0b2b60 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43064/0 (socket says 172.21.15.114:43064) 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 -- 172.21.15.114:0/550896401 learned_addr learned my addr 172.21.15.114:0/550896401 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51cb7fe700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51caffd700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51cc0b2b60 0x7f51cc160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 -- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51cc0b2b60 msgr2=0x7f51cc160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51cc0b2b60 0x7f51cc160ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 -- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51cc15e0f0 msgr2=0x7f51cc0c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.826+0000 7f51ca7fc700 1 -- 172.21.15.114:0/550896401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0094a0 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51ca7fc700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f51c00011e0 tx=0x7f51c000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0004c80 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c0004e00 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc1603c0 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0007310 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.827+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc1613e0 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.828+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc160bc0 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.828+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f51c0007470 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.829+0000 7f51b37fe700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 0x7f51b806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.829+0000 7f51caffd700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 0x7f51b806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.829+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51c0089310 con 0x7f51cc0cf2a0 2023-11-24T06:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.829+0000 7f51caffd700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 0x7f51b806d540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0090b0 tx=0x7f51bc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.844 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:17.833+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51cc160bc0 con 0x7f51cc0cf2a0 2023-11-24T06:28:18.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.025+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f51cc0d1400 con 0x7f51b806b080 2023-11-24T06:28:18.029 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.026+0000 7f51b37fe700 1 -- 172.21.15.114:0/550896401 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f51cc0d1400 con 0x7f51b806b080 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 msgr2=0x7f51b806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 0x7f51b806d540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0090b0 tx=0x7f51bc008030 comp rx=0 tx=0).stop 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 msgr2=0x7f51cc0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f51c00011e0 tx=0x7f51c000cae0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 shutdown_connections 2023-11-24T06:28:18.032 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51b806b080 0x7f51b806d540 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51cc15e0f0 0x7f51cc0c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51cc0b2b60 0x7f51cc160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 --2- 172.21.15.114:0/550896401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51cc0cf2a0 0x7f51cc0ce8d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 >> 172.21.15.114:0/550896401 conn(0x7f51cc0b5ce0 msgr2=0x7f51cc15d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 shutdown_connections 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.028+0000 7f51d17db700 1 -- 172.21.15.114:0/550896401 wait complete. 2023-11-24T06:28:18.033 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:28:18.043 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:28:18.043 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:18.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.165+0000 7f60281b9700 1 Processor -- start 2023-11-24T06:28:18.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.166+0000 7f60281b9700 1 -- start start 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.166+0000 7f60281b9700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60200b47e0 0x7f60200b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.166+0000 7f60281b9700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f602006b210 0x7f602006b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.166+0000 7f60281b9700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6020158be0 con 0x7f6020158800 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.166+0000 7f60281b9700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60200b51d0 con 0x7f60200b47e0 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f60281b9700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602006f9d0 con 0x7f602006b210 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60200b47e0 0x7f60200b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f60200b47e0 0x7f60200b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43072/0 (socket says 172.21.15.114:43072) 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 -- 172.21.15.114:0/450663377 learned_addr learned my addr 172.21.15.114:0/450663377 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6026756700 1 --2- 172.21.15.114:0/450663377 >> v2:172.21.15.172:3300/0 conn(0x7f602006b210 0x7f602006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 -- 172.21.15.114:0/450663377 >> v2:172.21.15.172:3300/0 conn(0x7f602006b210 msgr2=0x7f602006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 --2- 172.21.15.114:0/450663377 >> v2:172.21.15.172:3300/0 conn(0x7f602006b210 0x7f602006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 -- 172.21.15.114:0/450663377 >> v1:172.21.15.172:6789/0 conn(0x7f6020158800 legacy=0x7f602006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 -- 172.21.15.114:0/450663377 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602006edc0 con 0x7f60200b47e0 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6025754700 1 --2- 172.21.15.114:0/450663377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200b4c90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f60200d1810 tx=0x7f601c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af4ab1ffe4d41bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6024f53700 1 -- 172.21.15.114:0/450663377 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00b870 con 0x7f60200b47e0 2023-11-24T06:28:18.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6024f53700 1 -- 172.21.15.114:0/450663377 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00b9d0 con 0x7f60200b47e0 2023-11-24T06:28:18.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.167+0000 7f6024f53700 1 -- 172.21.15.114:0/450663377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00bd60 con 0x7f60200b47e0 2023-11-24T06:28:18.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 -- 172.21.15.114:0/450663377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 msgr2=0x7f60200b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 --2- 172.21.15.114:0/450663377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200b4c90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f60200d1810 tx=0x7f601c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 -- 172.21.15.114:0/450663377 shutdown_connections 2023-11-24T06:28:18.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 --2- 172.21.15.114:0/450663377 >> v2:172.21.15.172:3300/0 conn(0x7f602006b210 0x7f602006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 --2- 172.21.15.114:0/450663377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200b4c90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 -- 172.21.15.114:0/450663377 >> 172.21.15.114:0/450663377 conn(0x7f60200d0b40 msgr2=0x7f60200d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 -- 172.21.15.114:0/450663377 shutdown_connections 2023-11-24T06:28:18.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.168+0000 7f60281b9700 1 -- 172.21.15.114:0/450663377 wait complete. 2023-11-24T06:28:18.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.169+0000 7f60281b9700 1 Processor -- start 2023-11-24T06:28:18.178 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.169+0000 7f60281b9700 1 -- start start 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f602006b210 0x7f60200ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6020158800 0x7f60200ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60200aaa50 con 0x7f60200b47e0 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60200aae70 con 0x7f602006b210 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60200a9dd0 con 0x7f6020158800 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.179 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43078/0 (socket says 172.21.15.114:43078) 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 -- 172.21.15.114:0/814194442 learned_addr learned my addr 172.21.15.114:0/814194442 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025f55700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f602006b210 0x7f60200ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6026756700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6020158800 0x7f60200ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 -- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6020158800 msgr2=0x7f60200ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6020158800 0x7f60200ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 -- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f602006b210 msgr2=0x7f60200ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f602006b210 0x7f60200ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.180 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009550 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f6025754700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200a9460 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c004540 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c0036a0 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.170+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60200aa0b0 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.171+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c003a30 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.171+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60200aa670 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.171+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f602006f660 con 0x7f60200b47e0 2023-11-24T06:28:18.181 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f601c021070 con 0x7f60200b47e0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f600406b080 0x7f600406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f601c08ada0 con 0x7f60200b47e0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f6025f55700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f600406b080 0x7f600406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f6004070ee0 0x7f6004073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60040739d0 con 0x7f6004070ee0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.172+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f601c08d440 con 0x7f60200b47e0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.173+0000 7f6025f55700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f600406b080 0x7f600406d540 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.182 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.173+0000 7f6026756700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f6004070ee0 0x7f6004073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.173+0000 7f6026756700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f6004070ee0 0x7f6004073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.183 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.174+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== osd.6 v2:172.21.15.172:6821/1378310518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f60040739d0 con 0x7f6004070ee0 2023-11-24T06:28:18.192 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.189+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 --> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f602006f380 con 0x7f6004070ee0 2023-11-24T06:28:18.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.189+0000 7f60167fc700 1 -- 172.21.15.114:0/814194442 <== osd.6 v2:172.21.15.172:6821/1378310518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6018009b70 con 0x7f6004070ee0 2023-11-24T06:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:18.193 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.189+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f6004070ee0 msgr2=0x7f6004073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.189+0000 7f60281b9700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.172:6821/1378310518,v1:172.21.15.172:6823/1378310518] conn(0x7f6004070ee0 0x7f6004073300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f600406b080 msgr2=0x7f600406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f600406b080 0x7f600406d540 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 msgr2=0x7f60200a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 --2- 172.21.15.114:0/814194442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60200b47e0 0x7f60200a9460 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c004750 comp rx=0 tx=0).stop 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f6026756700 1 -- 172.21.15.114:0/814194442 reap_dead start 2023-11-24T06:28:18.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 shutdown_connections 2023-11-24T06:28:18.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 >> 172.21.15.114:0/814194442 conn(0x7f60200d0b40 msgr2=0x7f602006bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 shutdown_connections 2023-11-24T06:28:18.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.190+0000 7f60281b9700 1 -- 172.21.15.114:0/814194442 wait complete. 2023-11-24T06:28:18.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:28:18.325 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.322+0000 7fad8d16b700 1 Processor -- start 2023-11-24T06:28:18.326 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.322+0000 7fad8d16b700 1 -- start start 2023-11-24T06:28:18.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8d16b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 0x7fad880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8d16b700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880ae430 con 0x7fad8806c710 2023-11-24T06:28:18.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8d16b700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880ae710 con 0x7fad880af4d0 2023-11-24T06:28:18.328 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8d16b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880af0b0 con 0x7fad8815e080 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8759e700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fad8806c710 0x7fad8806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58798/0 (socket says 172.21.15.114:58798) 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad8759e700 1 -- 172.21.15.114:0/1108186029 learned_addr learned my addr 172.21.15.114:0/1108186029 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad86d9d700 1 --2- 172.21.15.114:0/1108186029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 0x7fad880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450709118 0 0) 0x7fad880ae430 con 0x7fad8806c710 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad700035b0 con 0x7fad8806c710 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad86d9d700 1 -- 172.21.15.114:0/1108186029 >> v1:172.21.15.172:6789/0 conn(0x7fad880af4d0 legacy=0x7fad8806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.329 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.323+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186712356 0 0) 0x7fad880ae710 con 0x7fad880af4d0 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad86d9d700 1 -- 172.21.15.114:0/1108186029 >> v1:172.21.15.114:6789/0 conn(0x7fad8806c710 legacy=0x7fad8806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad86d9d700 1 -- 172.21.15.114:0/1108186029 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880ae9f0 con 0x7fad8815e080 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad86d9d700 1 --2- 172.21.15.114:0/1108186029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 0x7fad880b5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c08fa64b2bc7bcb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011830 con 0x7fad8815e080 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c011990 con 0x7fad8815e080 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad85d9b700 1 -- 172.21.15.114:0/1108186029 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011d20 con 0x7fad8815e080 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 -- 172.21.15.114:0/1108186029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 msgr2=0x7fad880b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.330 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 --2- 172.21.15.114:0/1108186029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 0x7fad880b5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c009b80 comp rx=0 tx=0).stop 2023-11-24T06:28:18.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 -- 172.21.15.114:0/1108186029 shutdown_connections 2023-11-24T06:28:18.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 --2- 172.21.15.114:0/1108186029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8815e080 0x7fad880b5050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 -- 172.21.15.114:0/1108186029 >> 172.21.15.114:0/1108186029 conn(0x7fad88071b90 msgr2=0x7fad88071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.324+0000 7fad8d16b700 1 -- 172.21.15.114:0/1108186029 shutdown_connections 2023-11-24T06:28:18.331 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.325+0000 7fad8d16b700 1 -- 172.21.15.114:0/1108186029 wait complete. 2023-11-24T06:28:18.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.326+0000 7fad8d16b700 1 Processor -- start 2023-11-24T06:28:18.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.326+0000 7fad8d16b700 1 -- start start 2023-11-24T06:28:18.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8806c710 0x7fad880c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad880af4d0 0x7fad8806b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.335 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad880c64d0 con 0x7fad8815e080 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad880c67b0 con 0x7fad880af4d0 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880c6a90 con 0x7fad8806c710 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43096/0 (socket says 172.21.15.114:43096) 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 -- 172.21.15.114:0/2015469710 learned_addr learned my addr 172.21.15.114:0/2015469710 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:18.336 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad86d9d700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8806c710 0x7fad880c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8659c700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad880af4d0 0x7fad8806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 -- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8806c710 msgr2=0x7fad880c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8806c710 0x7fad880c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 -- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad880af4d0 msgr2=0x7fad8806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad880af4d0 0x7fad8806b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 -- 172.21.15.114:0/2015469710 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0094a0 con 0x7fad8815e080 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8759e700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fad8000cb40 tx=0x7fad8000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8000aaa0 con 0x7fad8815e080 2023-11-24T06:28:18.337 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad880c6d70 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8000ac00 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.327+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80012400 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.328+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad880c7050 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.328+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad880753f0 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.329+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fad8001b3b0 con 0x7fad8815e080 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.329+0000 7fad6f7fe700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 0x7fad7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.338 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.329+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad80089360 con 0x7fad8815e080 2023-11-24T06:28:18.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.329+0000 7fad86d9d700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 0x7fad7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.330+0000 7fad86d9d700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 0x7fad7406d540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0097c0 tx=0x7fad7c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.339 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.334+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad800544a0 con 0x7fad8815e080 2023-11-24T06:28:18.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.525+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fad880b05a0 con 0x7fad7406b080 2023-11-24T06:28:18.528 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.525+0000 7fad6f7fe700 1 -- 172.21.15.114:0/2015469710 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fad7c011b00 con 0x7fad7406b080 2023-11-24T06:28:18.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 msgr2=0x7fad7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.530 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 0x7fad7406d540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0097c0 tx=0x7fad7c01c040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 msgr2=0x7fad88158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fad8000cb40 tx=0x7fad8000ce50 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 shutdown_connections 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fad7406b080 0x7fad7406d540 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad8815e080 0x7fad88158710 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad8806c710 0x7fad880c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 --2- 172.21.15.114:0/2015469710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad880af4d0 0x7fad8806b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.531 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 >> 172.21.15.114:0/2015469710 conn(0x7fad88071b90 msgr2=0x7fad880b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 shutdown_connections 2023-11-24T06:28:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.527+0000 7fad8d16b700 1 -- 172.21.15.114:0/2015469710 wait complete. 2023-11-24T06:28:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:28:18.542 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:28:18.542 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:28:18.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.662+0000 7fc57d455700 1 Processor -- start 2023-11-24T06:28:18.666 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.662+0000 7fc57d455700 1 -- start start 2023-11-24T06:28:18.668 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc57d455700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc578071510 0x7fc5780cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc57d455700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc57806f550 0x7fc5780718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc57d455700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57815b350 con 0x7fc5780b2f60 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc57d455700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5780cf9c0 con 0x7fc578071510 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc57d455700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc578071e30 con 0x7fc57806f550 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc576ffd700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc578071510 0x7fc5780cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc5777fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc5780b2f60 0x7fc5780b33f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58810/0 (socket says 172.21.15.114:58810) 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.663+0000 7fc5777fe700 1 -- 172.21.15.114:0/3998431069 learned_addr learned my addr 172.21.15.114:0/3998431069 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:18.669 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc5767fc700 1 --2- 172.21.15.114:0/3998431069 >> v2:172.21.15.172:3300/0 conn(0x7fc57806f550 0x7fc5780718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc576ffd700 1 -- 172.21.15.114:0/3998431069 >> v2:172.21.15.172:3300/0 conn(0x7fc57806f550 msgr2=0x7fc5780718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3998431069 >> v2:172.21.15.172:3300/0 conn(0x7fc57806f550 0x7fc5780718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc576ffd700 1 -- 172.21.15.114:0/3998431069 >> v1:172.21.15.114:6789/0 conn(0x7fc5780b2f60 legacy=0x7fc5780b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc575ffb700 1 -- 172.21.15.114:0/3998431069 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566541215 0 0) 0x7fc57815b350 con 0x7fc5780b2f60 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc576ffd700 1 -- 172.21.15.114:0/3998431069 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780cfbd0 con 0x7fc578071510 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3998431069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc578071510 0x7fc5780cf480 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc5780b6220 tx=0x7fc56c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f652308c429dbcea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.670 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc575ffb700 1 -- 172.21.15.114:0/3998431069 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c011830 con 0x7fc578071510 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc575ffb700 1 -- 172.21.15.114:0/3998431069 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c011990 con 0x7fc578071510 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.664+0000 7fc575ffb700 1 -- 172.21.15.114:0/3998431069 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c011d20 con 0x7fc578071510 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 -- 172.21.15.114:0/3998431069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc578071510 msgr2=0x7fc5780cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 --2- 172.21.15.114:0/3998431069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc578071510 0x7fc5780cf480 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc5780b6220 tx=0x7fc56c009b20 comp rx=0 tx=0).stop 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 -- 172.21.15.114:0/3998431069 shutdown_connections 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 --2- 172.21.15.114:0/3998431069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc578071510 0x7fc5780cf480 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 --2- 172.21.15.114:0/3998431069 >> v2:172.21.15.172:3300/0 conn(0x7fc57806f550 0x7fc5780718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.671 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 -- 172.21.15.114:0/3998431069 >> 172.21.15.114:0/3998431069 conn(0x7fc5780cfef0 msgr2=0x7fc5780d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 -- 172.21.15.114:0/3998431069 shutdown_connections 2023-11-24T06:28:18.672 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.665+0000 7fc57d455700 1 -- 172.21.15.114:0/3998431069 wait complete. 2023-11-24T06:28:18.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.666+0000 7fc57d455700 1 Processor -- start 2023-11-24T06:28:18.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.666+0000 7fc57d455700 1 -- start start 2023-11-24T06:28:18.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57806f550 0x7fc57814a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc578071510 0x7fc57814fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.675 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 0x7fc57814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578137990 con 0x7fc5780b2f60 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc578137c70 con 0x7fc57806f550 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc57d455700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780c9480 con 0x7fc578071510 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 0x7fc57814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 0x7fc57814ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43106/0 (socket says 172.21.15.114:43106) 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 -- 172.21.15.114:0/3765992602 learned_addr learned my addr 172.21.15.114:0/3765992602 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5767fc700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc578071510 0x7fc57814fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.676 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57806f550 0x7fc57814a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 -- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc578071510 msgr2=0x7fc57814fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc578071510 0x7fc57814fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 -- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57806f550 msgr2=0x7fc57814a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57806f550 0x7fc57814a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0094a0 con 0x7fc5780b2f60 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5767fc700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc578071510 0x7fc57814fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57806f550 0x7fc57814a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:18.677 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.667+0000 7fc5777fe700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 0x7fc57814ff40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc57000cba0 tx=0x7fc57000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000ab80 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57000ad00 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5780c9760 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570007310 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc578136e10 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.668+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc578074ac0 con 0x7fc5780b2f60 2023-11-24T06:28:18.678 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.669+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc570007470 con 0x7fc5780b2f60 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.669+0000 7fc55f7fe700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc56406b080 0x7fc56406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.669+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc570089a90 con 0x7fc5780b2f60 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.669+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc56406b080 0x7fc56406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc55f7fe700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7fc564070ee0 0x7fc564073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5640739d0 con 0x7fc564070ee0 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc570089ee0 con 0x7fc5780b2f60 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc5767fc700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7fc564070ee0 0x7fc564073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.679 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc576ffd700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc56406b080 0x7fc56406d540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009b60 tx=0x7fc56c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.670+0000 7fc5767fc700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7fc564070ee0 0x7fc564073300 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.680 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.671+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== osd.7 v2:172.21.15.172:6808/2370047192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc5640739d0 con 0x7fc564070ee0 2023-11-24T06:28:18.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 --> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5780c95f0 con 0x7fc564070ee0 2023-11-24T06:28:18.689 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc55f7fe700 1 -- 172.21.15.114:0/3765992602 <== osd.7 v2:172.21.15.172:6808/2370047192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc5780c95f0 con 0x7fc564070ee0 2023-11-24T06:28:18.690 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2023-11-24T06:28:18.690 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:28:18.690 INFO:teuthology.orchestra.run.smithi114.stdout:} 2023-11-24T06:28:18.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7fc564070ee0 msgr2=0x7fc564073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.172:6808/2370047192,v1:172.21.15.172:6809/2370047192] conn(0x7fc564070ee0 0x7fc564073300 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.691 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc56406b080 msgr2=0x7fc56406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fc56406b080 0x7fc56406d540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009b60 tx=0x7fc56c008030 comp rx=0 tx=0).stop 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 msgr2=0x7fc57814ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc57d455700 1 --2- 172.21.15.114:0/3765992602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5780b2f60 0x7fc57814ff40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc57000cba0 tx=0x7fc57000cf60 comp rx=0 tx=0).stop 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.686+0000 7fc5777fe700 1 -- 172.21.15.114:0/3765992602 reap_dead start 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.687+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 shutdown_connections 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.687+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 >> 172.21.15.114:0/3765992602 conn(0x7fc5780cfef0 msgr2=0x7fc578159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.692 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.687+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 shutdown_connections 2023-11-24T06:28:18.693 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.687+0000 7fc57d455700 1 -- 172.21.15.114:0/3765992602 wait complete. 2023-11-24T06:28:18.702 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:28:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.820+0000 7f1796a54700 1 Processor -- start 2023-11-24T06:28:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.820+0000 7f1796a54700 1 -- start start 2023-11-24T06:28:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1796a54700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f17900d0d20 0x7f17900b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1796a54700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17901629e0 con 0x7f179015f0e0 2023-11-24T06:28:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1796a54700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1790161c60 con 0x7f179015ed00 2023-11-24T06:28:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1796a54700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17901625c0 con 0x7f17900d0d20 2023-11-24T06:28:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1794ff1700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f17900d0d20 0x7f17900b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178f7fe700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f179015f0e0 0x7f17900d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58822/0 (socket says 172.21.15.114:58822) 2023-11-24T06:28:18.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178f7fe700 1 -- 172.21.15.114:0/4074084048 learned_addr learned my addr 172.21.15.114:0/4074084048 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:18.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1794ff1700 1 -- 172.21.15.114:0/4074084048 >> v1:172.21.15.172:6789/0 conn(0x7f179015ed00 legacy=0x7f17900cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1794ff1700 1 -- 172.21.15.114:0/4074084048 >> v1:172.21.15.114:6789/0 conn(0x7f179015f0e0 legacy=0x7f17900d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1794ff1700 1 -- 172.21.15.114:0/4074084048 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900cf670 con 0x7f17900d0d20 2023-11-24T06:28:18.829 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178effd700 1 -- 172.21.15.114:0/4074084048 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087285702 0 0) 0x7f1784001110 con 0x7f179015f0e0 2023-11-24T06:28:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f1794ff1700 1 --2- 172.21.15.114:0/4074084048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17900d0d20 0x7f17900b90e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15948eeb42142a5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178effd700 1 -- 172.21.15.114:0/4074084048 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800e070 con 0x7f17900d0d20 2023-11-24T06:28:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178effd700 1 -- 172.21.15.114:0/4074084048 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178800bbd0 con 0x7f17900d0d20 2023-11-24T06:28:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.821+0000 7f178effd700 1 -- 172.21.15.114:0/4074084048 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788003a20 con 0x7f17900d0d20 2023-11-24T06:28:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 -- 172.21.15.114:0/4074084048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17900d0d20 msgr2=0x7f17900b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 --2- 172.21.15.114:0/4074084048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17900d0d20 0x7f17900b90e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178800b330 comp rx=0 tx=0).stop 2023-11-24T06:28:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 -- 172.21.15.114:0/4074084048 shutdown_connections 2023-11-24T06:28:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 --2- 172.21.15.114:0/4074084048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17900d0d20 0x7f17900b90e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 -- 172.21.15.114:0/4074084048 >> 172.21.15.114:0/4074084048 conn(0x7f179006b620 msgr2=0x7f1790071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 -- 172.21.15.114:0/4074084048 shutdown_connections 2023-11-24T06:28:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.822+0000 7f1796a54700 1 -- 172.21.15.114:0/4074084048 wait complete. 2023-11-24T06:28:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.823+0000 7f1796a54700 1 Processor -- start 2023-11-24T06:28:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.823+0000 7f1796a54700 1 -- start start 2023-11-24T06:28:18.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17900d0d20 0x7f17900cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f179015ed00 0x7f17900cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.836 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cb610 con 0x7f179015f0e0 2023-11-24T06:28:18.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cbfa0 con 0x7f17900d0d20 2023-11-24T06:28:18.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1796a54700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1790155c10 con 0x7f179015ed00 2023-11-24T06:28:18.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.837 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43122/0 (socket says 172.21.15.114:43122) 2023-11-24T06:28:18.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 -- 172.21.15.114:0/3160815153 learned_addr learned my addr 172.21.15.114:0/3160815153 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:18.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 -- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f179015ed00 msgr2=0x7f17900cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f179015ed00 0x7f17900cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 -- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17900d0d20 msgr2=0x7f17900cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.838 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17900d0d20 0x7f17900cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.824+0000 7f1794ff1700 1 -- 172.21.15.114:0/3160815153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009a70 con 0x7f179015f0e0 2023-11-24T06:28:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f1794ff1700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f178800b370 tx=0x7f1788004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788012840 con 0x7f179015f0e0 2023-11-24T06:28:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17880129a0 con 0x7f179015f0e0 2023-11-24T06:28:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790155ef0 con 0x7f179015f0e0 2023-11-24T06:28:18.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788012d50 con 0x7f179015f0e0 2023-11-24T06:28:18.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.825+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901564b0 con 0x7f179015f0e0 2023-11-24T06:28:18.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.826+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790156670 con 0x7f179015f0e0 2023-11-24T06:28:18.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.826+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f178802e420 con 0x7f179015f0e0 2023-11-24T06:28:18.840 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.827+0000 7f178cff9700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 0x7f177c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.827+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f178808c0c0 con 0x7f179015f0e0 2023-11-24T06:28:18.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.827+0000 7f178ffff700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 0x7f177c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.827+0000 7f178ffff700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 0x7f177c06d540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f17901e8470 tx=0x7f1780007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.841 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:18.830+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1788057250 con 0x7f179015f0e0 2023-11-24T06:28:19.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.019+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f17901621e0 con 0x7f177c06b080 2023-11-24T06:28:19.023 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.020+0000 7f178cff9700 1 -- 172.21.15.114:0/3160815153 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1780014400 con 0x7f177c06b080 2023-11-24T06:28:19.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 msgr2=0x7f177c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 0x7f177c06d540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f17901e8470 tx=0x7f1780007a40 comp rx=0 tx=0).stop 2023-11-24T06:28:19.026 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 msgr2=0x7f1790154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f178800b370 tx=0x7f1788004ad0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 shutdown_connections 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f177c06b080 0x7f177c06d540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f179015f0e0 0x7f1790154cc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17900d0d20 0x7f17900cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.027 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 --2- 172.21.15.114:0/3160815153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f179015ed00 0x7f17900cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 >> 172.21.15.114:0/3160815153 conn(0x7f179006b620 msgr2=0x7f179015c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 shutdown_connections 2023-11-24T06:28:19.028 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.022+0000 7f1796a54700 1 -- 172.21.15.114:0/3160815153 wait complete. 2023-11-24T06:28:19.028 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:28:19.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:19.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.155+0000 7f3641170700 1 Processor -- start 2023-11-24T06:28:19.159 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.155+0000 7f3641170700 1 -- start start 2023-11-24T06:28:19.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3641170700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 0x7f363c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3641170700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3641170700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0ae430 con 0x7f363c15e080 2023-11-24T06:28:19.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3641170700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0af0b0 con 0x7f363c06c710 2023-11-24T06:28:19.161 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3641170700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0b05a0 con 0x7f363c0af4d0 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f363b59e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f3633fff700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 0x7f363c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f363b59e700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.114:37676/0 (socket says 172.21.15.114:37676) 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.156+0000 7f363b59e700 1 -- 172.21.15.114:0/1399280876 learned_addr learned my addr 172.21.15.114:0/1399280876 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363b59e700 1 -- 172.21.15.114:0/1399280876 >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 msgr2=0x7f363c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363b59e700 1 --2- 172.21.15.114:0/1399280876 >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 0x7f363c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.162 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363b59e700 1 -- 172.21.15.114:0/1399280876 >> v1:172.21.15.172:6789/0 conn(0x7f363c15e080 legacy=0x7f363c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363b59e700 1 -- 172.21.15.114:0/1399280876 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0ae710 con 0x7f363c0af4d0 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f3633fff700 1 --2- 172.21.15.114:0/1399280876 >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 0x7f363c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363b59e700 1 --2- 172.21.15.114:0/1399280876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e50 tx=0x7f363400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab65efd6b55879b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363a59c700 1 -- 172.21.15.114:0/1399280876 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400e070 con 0x7f363c0af4d0 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363a59c700 1 -- 172.21.15.114:0/1399280876 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400b910 con 0x7f363c0af4d0 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.157+0000 7f363a59c700 1 -- 172.21.15.114:0/1399280876 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400bcc0 con 0x7f363c0af4d0 2023-11-24T06:28:19.163 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 -- 172.21.15.114:0/1399280876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 msgr2=0x7f363c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 --2- 172.21.15.114:0/1399280876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e50 tx=0x7f363400b330 comp rx=0 tx=0).stop 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 -- 172.21.15.114:0/1399280876 shutdown_connections 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 --2- 172.21.15.114:0/1399280876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c0af4d0 0x7f363c0afb90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 --2- 172.21.15.114:0/1399280876 >> v2:172.21.15.172:3300/0 conn(0x7f363c06c710 0x7f363c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 -- 172.21.15.114:0/1399280876 >> 172.21.15.114:0/1399280876 conn(0x7f363c071b90 msgr2=0x7f363c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 -- 172.21.15.114:0/1399280876 shutdown_connections 2023-11-24T06:28:19.164 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.158+0000 7f3641170700 1 -- 172.21.15.114:0/1399280876 wait complete. 2023-11-24T06:28:19.167 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.159+0000 7f3641170700 1 Processor -- start 2023-11-24T06:28:19.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.159+0000 7f3641170700 1 -- start start 2023-11-24T06:28:19.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c06c710 0x7f363c1464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.169 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f363c15e080 0x7f363c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c15e500 con 0x7f363c0af4d0 2023-11-24T06:28:19.170 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c146a20 con 0x7f363c15e080 2023-11-24T06:28:19.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3641170700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c155310 con 0x7f363c06c710 2023-11-24T06:28:19.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43136/0 (socket says 172.21.15.114:43136) 2023-11-24T06:28:19.171 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 -- 172.21.15.114:0/1899475860 learned_addr learned my addr 172.21.15.114:0/1899475860 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:19.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f363ad9d700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c06c710 0x7f363c1464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f363b59e700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f363c15e080 0x7f363c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 -- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c06c710 msgr2=0x7f363c1464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.172 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c06c710 0x7f363c1464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 -- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f363c15e080 msgr2=0x7f363c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f363c15e080 0x7f363c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 -- 172.21.15.114:0/1899475860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240094a0 con 0x7f363c0af4d0 2023-11-24T06:28:19.173 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.160+0000 7f3633fff700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f362400ebe0 tx=0x7f362400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400c850 con 0x7f363c0af4d0 2023-11-24T06:28:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400c9b0 con 0x7f363c0af4d0 2023-11-24T06:28:19.174 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3634009b30 con 0x7f363c0af4d0 2023-11-24T06:28:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362400cd40 con 0x7f363c0af4d0 2023-11-24T06:28:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c15e770 con 0x7f363c0af4d0 2023-11-24T06:28:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.161+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0753f0 con 0x7f363c0af4d0 2023-11-24T06:28:19.175 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.162+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3624013470 con 0x7f363c0af4d0 2023-11-24T06:28:19.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.162+0000 7f36337fe700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 0x7f361c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.163+0000 7f363ad9d700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 0x7f361c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.163+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3624089720 con 0x7f363c0af4d0 2023-11-24T06:28:19.176 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.163+0000 7f363ad9d700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 0x7f361c06d540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f362c007fd0 tx=0x7f362c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.177 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.167+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36240547d0 con 0x7f363c0af4d0 2023-11-24T06:28:19.356 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.353+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f363c061a10 con 0x7f361c06b080 2023-11-24T06:28:19.358 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.355+0000 7f36337fe700 1 -- 172.21.15.114:0/1899475860 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50689 (secure 0 0 0) 0x7f363c061a10 con 0x7f361c06b080 2023-11-24T06:28:19.359 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:19.360 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 msgr2=0x7f361c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 0x7f361c06d540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f362c007fd0 tx=0x7f362c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 msgr2=0x7f363c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f362400ebe0 tx=0x7f362400eef0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 shutdown_connections 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f361c06b080 0x7f361c06d540 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f363c15e080 0x7f363c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.361 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f363c06c710 0x7f363c1464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 --2- 172.21.15.114:0/1899475860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f363c0af4d0 0x7f363c15fd90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 >> 172.21.15.114:0/1899475860 conn(0x7f363c071b90 msgr2=0x7f363c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 shutdown_connections 2023-11-24T06:28:19.362 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:19.357+0000 7f3641170700 1 -- 172.21.15.114:0/1899475860 wait complete. 2023-11-24T06:28:19.362 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:28:19.373 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2023-11-24T06:28:17.759160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":223144,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751576,"statfs":{"total":773094113280,"available":772865613824,"internally_reserved":0,"allocated":7118848,"data_stored":3640304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.300787"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T06:28:09.756498+0000","last_change":"2023-11-24T06:24:06.944365+0000","last_active":"2023-11-24T06:28:09.756498+0000","last_peered":"2023-11-24T06:28:09.756498+0000","last_clean":"2023-11-24T06:28:09.756498+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:28:09.756498+0000","last_undegraded":"2023-11-24T06:28:09.756498+0000","last_fullsized":"2023-11-24T06:28:09.756498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:24:05.800491+0000","last_clean_scrub_stamp":"2023-11-24T06:24:05.800491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:19.774863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":1335296,"data_stored":897814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.65500000000000003,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":4.407,"15min":4.407},"last":0.40999999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.083,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.746,"15min":0.91400000000000003},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0569999999999999,"5min":1.232,"15min":1.232},"last":0.376},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.73799999999999999,"15min":0.91000000000000003},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.316},{"interface":"front","average":{"1min":0.62,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.997,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.53000000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.40500000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81399999999999995,"5min":1.131,"15min":1.131},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":1.079,"15min":1.079},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.753,"5min":1.042,"15min":1.042},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.746,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92900000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71199999999999997,"15min":0.84499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.294,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":1.294},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":3.427}]},{"osd":2,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.411,"15min":1.411},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.345,"15min":1.345},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":3.5209999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.7730000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54800000000000004,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":1.379,"15min":1.379},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.50800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.45400000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.37},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.35499999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.224}]},{"osd":2,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.23000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.26100000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.371,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.47299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40200000000000002,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54500000000000004,"5min":0.75,"15min":0.75},"last":0.29099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64700000000000002,"15min":0.67500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.498,"5min":0.372,"15min":0.372},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.63200000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.62},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86799999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.46000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":1335296,"data_stored":897814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.64900000000000002,"15min":0.96699999999999997},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.48899999999999999,"5min":11.44,"15min":11.44},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.45700000000000002,"15min":0.56000000000000005},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54900000000000004,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.52500000000000002,"15min":0.64500000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.48599999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.68799999999999994,"15min":0.99399999999999999},"min":{"1min":0.27000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52900000000000003,"5min":11.157,"15min":11.157},"last":0.32500000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.53500000000000003,"15min":0.64700000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59699999999999998,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.48199999999999998,"15min":0.57199999999999995},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.51900000000000002,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.26500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.90300000000000002,"15min":1.2290000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":11.378,"15min":11.378},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.69599999999999995,"15min":0.81100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81200000000000006,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.75}]},{"osd":5,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.66400000000000003,"15min":0.71899999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":1.728,"15min":1.728},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.752,"15min":0.874},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.65400000000000003,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80200000000000005,"5min":1.7,"15min":1.7},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.92500000000000004,"15min":1.2509999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":11.343,"15min":11.343},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.96399999999999997,"15min":1.284},"min":{"1min":0.45900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.81799999999999995,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.312,"15min":0.312},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.436,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51300000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.315},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55700000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53900000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81200000000000006,"5min":1.028,"15min":1.028},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85899999999999999,"5min":1.01,"15min":1.01},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63300000000000001,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.435},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63200000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52000000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.004,"15min":1.004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":1,"15min":1},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":1.181,"15min":1.181},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.65400000000000003,"15min":0.78600000000000003},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.503,"5min":10.052,"15min":10.052},"last":0.441}]},{"osd":1,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41099999999999998,"15min":0.437},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.25700000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.32100000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60399999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.69199999999999995,"15min":0.82199999999999995},"min":{"1min":0.34300000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58599999999999997,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.505}]},{"osd":4,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":1.306,"15min":1.306},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.90400000000000003,"15min":1.0409999999999999},"min":{"1min":0.40899999999999997,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.496,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.89300000000000002,"15min":1.0169999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82399999999999995,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.73799999999999999,"15min":0.79600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86799999999999999,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82099999999999995,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.016,"15min":1.1919999999999999},"min":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89500000000000002,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.91600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":2.169,"15min":2.169},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.46500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.497},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73699999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.63800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79400000000000004,"5min":1.351,"15min":1.351},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.62}]},{"osd":6,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56999999999999995,"15min":0.59299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.751,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.52000000000000002},"min":{"1min":0.45700000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.48899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55300000000000005,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82999999999999996,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.52300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":1.397,"15min":1.397},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51500000000000001,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56100000000000005,"15min":0.57099999999999995},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.495,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84099999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82799999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.47899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:19.377 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:24:05.800491+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=24, tm_sec=5, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=28, tm_sec=14, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:28:19.377 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:28:39.378 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.510+0000 7f791207f700 1 Processor -- start 2023-11-24T06:28:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.510+0000 7f791207f700 1 -- start start 2023-11-24T06:28:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f791207f700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 0x7f790c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f791207f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c15eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f791207f700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c15f520 con 0x7f790c06c000 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f791207f700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790c0b4230 con 0x7f790c144660 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f791207f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0b3c40 con 0x7f790c15ea60 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790affd700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c15eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790b7fe700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 0x7f790c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790bfff700 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f790c06c000 0x7f790c06ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.114:52450/0 (socket says 172.21.15.114:52450) 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790bfff700 1 -- 172.21.15.114:0/3068094536 learned_addr learned my addr 172.21.15.114:0/3068094536 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790a7fc700 1 -- 172.21.15.114:0/3068094536 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266965168 0 0) 0x7f790c15f520 con 0x7f790c06c000 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790affd700 1 -- 172.21.15.114:0/3068094536 >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 msgr2=0x7f790c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790affd700 1 --2- 172.21.15.114:0/3068094536 >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 0x7f790c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.511+0000 7f790affd700 1 -- 172.21.15.114:0/3068094536 >> v1:172.21.15.172:6789/0 conn(0x7f790c06c000 legacy=0x7f790c06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790affd700 1 -- 172.21.15.114:0/3068094536 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c15f800 con 0x7f790c15ea60 2023-11-24T06:28:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790b7fe700 1 --2- 172.21.15.114:0/3068094536 >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 0x7f790c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790affd700 1 --2- 172.21.15.114:0/3068094536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c15eec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009840 tx=0x7f78fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21644a32f8f5efa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790a7fc700 1 -- 172.21.15.114:0/3068094536 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc011830 con 0x7f790c15ea60 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790a7fc700 1 -- 172.21.15.114:0/3068094536 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc011990 con 0x7f790c15ea60 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f790a7fc700 1 -- 172.21.15.114:0/3068094536 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc011d20 con 0x7f790c15ea60 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 -- 172.21.15.114:0/3068094536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 msgr2=0x7f790c15eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 --2- 172.21.15.114:0/3068094536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c15eec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009840 tx=0x7f78fc009b50 comp rx=0 tx=0).stop 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 -- 172.21.15.114:0/3068094536 shutdown_connections 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 --2- 172.21.15.114:0/3068094536 >> v2:172.21.15.172:3300/0 conn(0x7f790c144660 0x7f790c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 --2- 172.21.15.114:0/3068094536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c15eec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 -- 172.21.15.114:0/3068094536 >> 172.21.15.114:0/3068094536 conn(0x7f790c071ac0 msgr2=0x7f790c071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 -- 172.21.15.114:0/3068094536 shutdown_connections 2023-11-24T06:28:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.512+0000 7f791207f700 1 -- 172.21.15.114:0/3068094536 wait complete. 2023-11-24T06:28:39.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.514+0000 7f791207f700 1 Processor -- start 2023-11-24T06:28:39.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.514+0000 7f791207f700 1 -- start start 2023-11-24T06:28:39.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f790c06c000 0x7f790c139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.523 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c1f8560 con 0x7f790c144660 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c140880 con 0x7f790c06c000 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c140b60 con 0x7f790c15ea60 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43150/0 (socket says 172.21.15.114:43150) 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 -- 172.21.15.114:0/1858580607 learned_addr learned my addr 172.21.15.114:0/1858580607 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790b7fe700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f790c06c000 0x7f790c139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.524 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790bfff700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 -- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 msgr2=0x7f790c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c06fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 -- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f790c06c000 msgr2=0x7f790c139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f790c06c000 0x7f790c139c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 -- 172.21.15.114:0/1858580607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79000094a0 con 0x7f790c144660 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f790affd700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009fd0 tx=0x7f78fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0036a0 con 0x7f790c144660 2023-11-24T06:28:39.525 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc01f070 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0094a0 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc003a60 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.515+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c141120 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.516+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c0b5690 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.517+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f78fc003bc0 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.517+0000 7f78f3fff700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 0x7f78f806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.517+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78fc021070 con 0x7f790c144660 2023-11-24T06:28:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.517+0000 7f790b7fe700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 0x7f78f806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.518+0000 7f790b7fe700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 0x7f78f806d540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7900009d70 tx=0x7f7900009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.522+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78fc055f40 con 0x7f790c144660 2023-11-24T06:28:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.707+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 --> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f790c13aec0 con 0x7f78f806b080 2023-11-24T06:28:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.709+0000 7f78f3fff700 1 -- 172.21.15.114:0/1858580607 <== mgr.4107 v2:172.21.15.114:6832/407740377 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50427 (secure 0 0 0) 0x7f790c13aec0 con 0x7f78f806b080 2023-11-24T06:28:39.713 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 msgr2=0x7f78f806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 0x7f78f806d540 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7900009d70 tx=0x7f7900009210 comp rx=0 tx=0).stop 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 msgr2=0x7f790c06f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009fd0 tx=0x7f78fc004750 comp rx=0 tx=0).stop 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 shutdown_connections 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f78f806b080 0x7f78f806d540 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.715 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f790c15ea60 0x7f790c06fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f790c06c000 0x7f790c139c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 --2- 172.21.15.114:0/1858580607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f790c144660 0x7f790c06f880 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 >> 172.21.15.114:0/1858580607 conn(0x7f790c071ac0 msgr2=0x7f790c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 shutdown_connections 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.711+0000 7f791207f700 1 -- 172.21.15.114:0/1858580607 wait complete. 2023-11-24T06:28:39.716 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2023-11-24T06:28:39.727 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2023-11-24T06:28:37.762881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":223128,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751592,"statfs":{"total":773094113280,"available":772865630208,"internally_reserved":0,"allocated":7118848,"data_stored":3640304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002200"},"pg_stats":[{"pgid":"1.0","version":"61'119","reported_seq":218,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-24T06:28:24.615040+0000","last_change":"2023-11-24T06:28:24.615076+0000","last_active":"2023-11-24T06:28:24.615040+0000","last_peered":"2023-11-24T06:28:24.615040+0000","last_clean":"2023-11-24T06:28:24.615040+0000","last_became_active":"2023-11-24T06:24:06.943369+0000","last_became_peered":"2023-11-24T06:24:06.943369+0000","last_unstale":"2023-11-24T06:28:24.615040+0000","last_undegraded":"2023-11-24T06:28:24.615040+0000","last_fullsized":"2023-11-24T06:28:24.615040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'119","last_scrub_stamp":"2023-11-24T06:28:24.614950+0000","last_deep_scrub":"61'119","last_deep_scrub_stamp":"2023-11-24T06:28:24.614950+0000","last_clean_scrub_stamp":"2023-11-24T06:28:24.614950+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:28.727712+0000","scrub_duration":0.0061299450000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1335296,"data_stored":897814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.75,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.65500000000000003,"15min":0.73099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":4.407,"15min":4.407},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.083,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.746,"15min":0.91400000000000003},"min":{"1min":0.36199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80800000000000005,"5min":7.9340000000000002,"15min":7.9340000000000002},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0569999999999999,"5min":1.232,"15min":1.232},"last":0.501},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.73799999999999999,"15min":0.91000000000000003},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0369999999999999,"5min":8.5890000000000004,"15min":8.5890000000000004},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.997,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.53000000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52100000000000002,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40300000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.40500000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81399999999999995,"5min":1.131,"15min":1.131},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":1.079,"15min":1.079},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.753,"5min":1.042,"15min":1.042},"last":0.437},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.746,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.34999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92900000000000005,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71199999999999997,"15min":0.84499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.294,"5min":6.9189999999999996,"15min":6.9189999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97499999999999998,"5min":1.411,"15min":1.411},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86299999999999999,"5min":1.345,"15min":1.345},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.371,"15min":0.371},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54800000000000004,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":1.379,"15min":1.379},"last":0.621}]},{"osd":7,"last update":"Fri Nov 24 06:28:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.45400000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.35499999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.441,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.376}]},{"osd":2,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"min":{"1min":0.23000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.38,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51400000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.371,"15min":0.37},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.47299999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40200000000000002,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54500000000000004,"5min":0.75,"15min":0.75},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64700000000000002,"15min":0.67500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82199999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64500000000000002,"15min":0.66400000000000003},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.498,"5min":0.372,"15min":0.372},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83699999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.627,"15min":0.64900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84899999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:28:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61899999999999999,"15min":0.63200000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86799999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.68000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1335296,"data_stored":897814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.64900000000000002,"15min":0.96699999999999997},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.48899999999999999,"5min":11.44,"15min":11.44},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.45700000000000002,"15min":0.56000000000000005},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54900000000000004,"5min":3.0449999999999999,"15min":3.0449999999999999},"last":0.315}]},{"osd":2,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.52500000000000002,"15min":0.64500000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.48599999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.378},{"interface":"front","average":{"1min":0.373,"5min":0.68799999999999994,"15min":0.99399999999999999},"min":{"1min":0.27000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52900000000000003,"5min":11.157,"15min":11.157},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.53500000000000003,"15min":0.64700000000000002},"min":{"1min":0.29199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59699999999999998,"5min":3.4630000000000001,"15min":3.4630000000000001},"last":0.436},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.48199999999999998,"15min":0.57199999999999995},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.51900000000000002,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.90300000000000002,"15min":1.2290000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74299999999999999,"5min":11.378,"15min":11.378},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.69599999999999995,"15min":0.81100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81200000000000006,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.66400000000000003,"15min":0.71899999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84299999999999997,"5min":1.728,"15min":1.728},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.752,"15min":0.874},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79300000000000004,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.65400000000000003,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80200000000000005,"5min":1.7,"15min":1.7},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.92500000000000004,"15min":1.2509999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82899999999999996,"5min":11.343,"15min":11.343},"last":0.629}]},{"osd":7,"last update":"Fri Nov 24 06:28:11 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.96399999999999997,"15min":1.284},"min":{"1min":0.45900000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":11.407999999999999,"15min":11.407999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68200000000000005,"15min":0.75},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.81799999999999995,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.312,"15min":0.312},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.436,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51300000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53000000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55700000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.433}]},{"osd":3,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53900000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81200000000000006,"5min":1.028,"15min":1.028},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66100000000000003,"15min":0.68899999999999995},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82899999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85899999999999999,"5min":1.01,"15min":1.01},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63300000000000001,"15min":0.67300000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63200000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83599999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:28:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63200000000000001,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.004,"15min":1.004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79900000000000004,"5min":1,"15min":1},"last":0.52100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35099999999999998,"15min":0.374},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.57899999999999996,"5min":1.181,"15min":1.181},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.55000000000000004,"15min":0.752},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":10.052,"15min":10.052},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.38700000000000001,"15min":0.42899999999999999},"min":{"1min":0.23799999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53200000000000003,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39500000000000002,"15min":0.41899999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.55000000000000004,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41399999999999998,"15min":0.436},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.60499999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.59199999999999997,"15min":0.78800000000000003},"min":{"1min":0.27100000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57699999999999996,"5min":10.077999999999999,"15min":10.077999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":1.306,"15min":1.306},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.80600000000000005,"15min":1.008},"min":{"1min":0.42799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.81100000000000005,"5min":9.9459999999999997,"15min":9.9459999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.64500000000000002,"15min":0.68999999999999995},"min":{"1min":0.45600000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.93000000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.69199999999999995,"15min":0.77600000000000002},"min":{"1min":0.50800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83199999999999996,"5min":2.7919999999999998,"15min":2.7919999999999998},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.80500000000000005,"15min":0.98799999999999999},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81299999999999994,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70099999999999996,"15min":0.78300000000000003},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89600000000000002,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.88800000000000001,"15min":1.149},"min":{"1min":0.44800000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":9.9939999999999998,"15min":9.9939999999999998},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":2.169,"15min":2.169},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56799999999999995,"15min":0.55800000000000005},"min":{"1min":0.46500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72099999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79800000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.53600000000000003},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73699999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.63800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79400000000000004,"5min":1.351,"15min":1.351},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70199999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56999999999999995,"15min":0.59299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.751,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.374},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45500000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:28:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27748,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":741376,"data_stored":307446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.745,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.622},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82599999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":1.397,"15min":1.397},"last":0.5}]},{"osd":4,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51000000000000001,"15min":0.52400000000000002},"min":{"1min":0.28899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.70699999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79400000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.437,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77700000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.746,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:39.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:28:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.847+0000 7f51c56ed700 1 Processor -- start 2023-11-24T06:28:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 -- start start 2023-11-24T06:28:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51c00cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f51c00cd890 0x7f51c00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0160bc0 con 0x7f51c015f7c0 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c00cf700 con 0x7f51c00cd890 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51c56ed700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cd400 con 0x7f51c0160420 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51beffd700 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f51c015f7c0 0x7f51c015fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58864/0 (socket says 172.21.15.114:58864) 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.848+0000 7f51beffd700 1 -- 172.21.15.114:0/1542625694 learned_addr learned my addr 172.21.15.114:0/1542625694 (peer_addr_for_me v1:172.21.15.114:0/0) 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474745345 0 0) 0x7f51c0160bc0 con 0x7f51c015f7c0 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a80035b0 con 0x7f51c015f7c0 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bf7fe700 1 --2- 172.21.15.114:0/1542625694 >> v2:172.21.15.172:3300/0 conn(0x7f51c00cd890 0x7f51c00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51c00cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172654301 0 0) 0x7f51a80035b0 con 0x7f51c015f7c0 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 msgr2=0x7f51c00cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51c00cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 >> v2:172.21.15.172:3300/0 conn(0x7f51c00cd890 msgr2=0x7f51c00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 --2- 172.21.15.114:0/1542625694 >> v2:172.21.15.172:3300/0 conn(0x7f51c00cd890 0x7f51c00b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00ccad0 con 0x7f51c015f7c0 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1175656184 0 0) 0x7f51b4001670 con 0x7f51c015f7c0 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 >> v1:172.21.15.114:6789/0 conn(0x7f51c015f7c0 legacy=0x7f51c015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8003a00 0x7f51a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51a8004850 0x7f51a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cf700 con 0x7f51a8004850 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cd400 con 0x7f51a8003a00 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a800ba10 con 0x7f51c0160420 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51a8004850 0x7f51a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51beffd700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8003a00 0x7f51a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51bf7fe700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 -- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 msgr2=0x7f51a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c0160420 0x7f51a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 -- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8003a00 msgr2=0x7f51a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8003a00 0x7f51a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.849+0000 7f51be7fc700 1 -- 172.21.15.114:0/1542625694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80035b0 con 0x7f51a8004850 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51be7fc700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51a8004850 0x7f51a8004cd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b60 tx=0x7f51a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13239e92c2a5f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000e6c0 con 0x7f51a8004850 2023-11-24T06:28:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b000e820 con 0x7f51a8004850 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51bdffb700 1 -- 172.21.15.114:0/1542625694 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000eb90 con 0x7f51a8004850 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 -- 172.21.15.114:0/1542625694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51a8004850 msgr2=0x7f51a8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 --2- 172.21.15.114:0/1542625694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51a8004850 0x7f51a8004cd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009b60 tx=0x7f51a8005880 comp rx=0 tx=0).stop 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51bf7fe700 1 -- 172.21.15.114:0/1542625694 reap_dead start 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 -- 172.21.15.114:0/1542625694 shutdown_connections 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 -- 172.21.15.114:0/1542625694 >> 172.21.15.114:0/1542625694 conn(0x7f51c00b3810 msgr2=0x7f51c00b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 -- 172.21.15.114:0/1542625694 shutdown_connections 2023-11-24T06:28:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.850+0000 7f51c56ed700 1 -- 172.21.15.114:0/1542625694 wait complete. 2023-11-24T06:28:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 Processor -- start 2023-11-24T06:28:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 -- start start 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 0x7f51c00c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c00b5050 0x7f51c00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c015cad0 con 0x7f51c0156fe0 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c9c20 con 0x7f51a8004850 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.852+0000 7f51c56ed700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c9e30 con 0x7f51c00b5050 2023-11-24T06:28:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51beffd700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 0x7f51c00c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43172/0 (socket says 172.21.15.114:43172) 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 -- 172.21.15.114:0/2618522459 learned_addr learned my addr 172.21.15.114:0/2618522459 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51bf7fe700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c00b5050 0x7f51c00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 -- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c00b5050 msgr2=0x7f51c00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c00b5050 0x7f51c00cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 -- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 msgr2=0x7f51c00c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 0x7f51c00c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 -- 172.21.15.114:0/2618522459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c015ccc0 con 0x7f51c0156fe0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51beffd700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 0x7f51c00c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51be7fc700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f51b000e4b0 tx=0x7f51b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000e6c0 con 0x7f51c0156fe0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b00038c0 con 0x7f51c0156fe0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c015cf60 con 0x7f51c0156fe0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003c70 con 0x7f51c0156fe0 2023-11-24T06:28:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.853+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c015d520 con 0x7f51c0156fe0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.854+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c015d6e0 con 0x7f51c0156fe0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.855+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f51b0013470 con 0x7f51c0156fe0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.855+0000 7f51a77fe700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 0x7f51ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.855+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51b008ade0 con 0x7f51c0156fe0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.855+0000 7f51beffd700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 0x7f51ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.855+0000 7f51beffd700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 0x7f51ac06d540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51b4001820 tx=0x7f51b4007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:39.859+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51b00560c0 con 0x7f51c0156fe0 2023-11-24T06:28:40.051 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.048+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f51c015d890 con 0x7f51c0156fe0 2023-11-24T06:28:40.055 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.051+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f51b00596e0 con 0x7f51c0156fe0 2023-11-24T06:28:40.056 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.053+0000 7f51a77fe700 1 -- 172.21.15.114:0/2618522459 <== mon.0 v2:172.21.15.114:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f51b00560c0 con 0x7f51c0156fe0 2023-11-24T06:28:40.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 msgr2=0x7f51ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 0x7f51ac06d540 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51b4001820 tx=0x7f51b4007980 comp rx=0 tx=0).stop 2023-11-24T06:28:40.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 msgr2=0x7f51c0156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.058 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f51b000e4b0 tx=0x7f51b00047c0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 shutdown_connections 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7f51ac06b080 0x7f51ac06d540 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51c00b5050 0x7f51c00cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51a8004850 0x7f51c00c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 --2- 172.21.15.114:0/2618522459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51c0156fe0 0x7f51c0156900 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 >> 172.21.15.114:0/2618522459 conn(0x7f51c00b3810 msgr2=0x7f51c006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 shutdown_connections 2023-11-24T06:28:40.059 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.054+0000 7f51c56ed700 1 -- 172.21.15.114:0/2618522459 wait complete. 2023-11-24T06:28:40.070 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:28:40.071 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:40.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 Processor -- start 2023-11-24T06:28:40.191 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 -- start start 2023-11-24T06:28:40.193 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad40b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad41603c0 0x7fbad40d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad40d0770 con 0x7fbad40d1a50 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad415c2b0 con 0x7fbad41603c0 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.188+0000 7fbadad4a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40cd100 con 0x7fbad415bed0 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad41603c0 0x7fbad40d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad41603c0 0x7fbad40d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43178/0 (socket says 172.21.15.114:43178) 2023-11-24T06:28:40.194 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 -- 172.21.15.114:0/3842969325 learned_addr learned my addr 172.21.15.114:0/3842969325 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad8ae6700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad40b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 -- 172.21.15.114:0/3842969325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 msgr2=0x7fbad40b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad40b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 -- 172.21.15.114:0/3842969325 >> v1:172.21.15.172:6789/0 conn(0x7fbad40d1a50 legacy=0x7fbad40d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 -- 172.21.15.114:0/3842969325 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40d09a0 con 0x7fbad41603c0 2023-11-24T06:28:40.195 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad3fff700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40d0230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbac00098f0 tx=0x7fbac0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42bcabbdd694249 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad37fe700 1 -- 172.21.15.114:0/3842969325 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0011830 con 0x7fbad41603c0 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad37fe700 1 -- 172.21.15.114:0/3842969325 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbac0011990 con 0x7fbad41603c0 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.189+0000 7fbad37fe700 1 -- 172.21.15.114:0/3842969325 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0011d40 con 0x7fbad41603c0 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 -- 172.21.15.114:0/3842969325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 msgr2=0x7fbad40d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40d0230 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbac00098f0 tx=0x7fbac0009c00 comp rx=0 tx=0).stop 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 -- 172.21.15.114:0/3842969325 shutdown_connections 2023-11-24T06:28:40.196 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad40b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 --2- 172.21.15.114:0/3842969325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40d0230 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 -- 172.21.15.114:0/3842969325 >> 172.21.15.114:0/3842969325 conn(0x7fbad4073130 msgr2=0x7fbad409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 -- 172.21.15.114:0/3842969325 shutdown_connections 2023-11-24T06:28:40.197 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.190+0000 7fbadad4a700 1 -- 172.21.15.114:0/3842969325 wait complete. 2023-11-24T06:28:40.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.191+0000 7fbadad4a700 1 Processor -- start 2023-11-24T06:28:40.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.191+0000 7fbadad4a700 1 -- start start 2023-11-24T06:28:40.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbad40d1a50 0x7fbad4159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.200 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41f1e20 con 0x7fbad41603c0 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad415dd80 con 0x7fbad40d1a50 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbadad4a700 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad415d4e0 con 0x7fbad415bed0 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43186/0 (socket says 172.21.15.114:43186) 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 -- 172.21.15.114:0/1959727452 learned_addr learned my addr 172.21.15.114:0/1959727452 (peer_addr_for_me v2:172.21.15.114:0/0) 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad8ae6700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbad40d1a50 0x7fbad4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.201 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad3fff700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 -- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 msgr2=0x7fbad415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 -- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbad40d1a50 msgr2=0x7fbad4159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbad40d1a50 0x7fbad4159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 -- 172.21.15.114:0/1959727452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0009550 con 0x7fbad41603c0 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad92e7700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00cbc0 tx=0x7fbacc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.202 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.192+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc003c00 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.193+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbacc003d60 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.193+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad415d7c0 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.193+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ac10 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.193+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad415e650 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.193+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad4067f60 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.194+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbacc0050d0 con 0x7fbad41603c0 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.194+0000 7fbad17fa700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 0x7fbab806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.203 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.194+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbacc0897d0 con 0x7fbad41603c0 2023-11-24T06:28:40.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.194+0000 7fbad8ae6700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 0x7fbab806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.195+0000 7fbad8ae6700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 0x7fbab806d540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009b70 tx=0x7fbac8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.204 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.199+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbacc054870 con 0x7fbad41603c0 2023-11-24T06:28:40.434 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.431+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbad41f1fb0 con 0x7fbad41603c0 2023-11-24T06:28:40.435 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.431+0000 7fbad17fa700 1 -- 172.21.15.114:0/1959727452 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fbacc057e90 con 0x7fbad41603c0 2023-11-24T06:28:40.435 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-24T06:28:40.436 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1063420043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1063420043},{"type":"v1","addr":"172.21.15.172:6835","nonce":1063420043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1403055793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1403055793},{"type":"v1","addr":"172.21.15.172:6834","nonce":1403055793}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6836/3135080513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3135080513},{"type":"v1","addr":"172.21.15.114:6836","nonce":3135080513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6555,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2821599752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6835","nonce":2821599752},{"type":"v1","addr":"172.21.15.114:6837","nonce":2821599752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[]} 2023-11-24T06:28:40.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 msgr2=0x7fbab806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 0x7fbab806d540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009b70 tx=0x7fbac8008040 comp rx=0 tx=0).stop 2023-11-24T06:28:40.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 msgr2=0x7fbad40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.437 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00cbc0 tx=0x7fbacc00cf80 comp rx=0 tx=0).stop 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 shutdown_connections 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:6832/407740377,v1:172.21.15.114:6833/407740377] conn(0x7fbab806b080 0x7fbab806d540 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad41603c0 0x7fbad40caf20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbad40d1a50 0x7fbad4159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 --2- 172.21.15.114:0/1959727452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbad415bed0 0x7fbad415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.433+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 >> 172.21.15.114:0/1959727452 conn(0x7fbad4073130 msgr2=0x7fbad415f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.438 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.434+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 shutdown_connections 2023-11-24T06:28:40.439 INFO:teuthology.orchestra.run.smithi114.stderr:2023-11-24T06:28:40.434+0000 7fbadad4a700 1 -- 172.21.15.114:0/1959727452 wait complete. 2023-11-24T06:28:40.439 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 40 2023-11-24T06:28:40.449 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T06:28:40.449 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:28:40.449 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:28:40.450 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.458 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:28:40.458 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:28:40.458 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.507 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:28:40.507 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:28:40.509 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.519 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:28:40.519 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:28:40.519 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.570 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:28:40.571 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:28:40.571 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:28:40.571 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.601 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:28:40.602 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:28:40.602 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.675 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:28:40.676 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:28:40.676 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.742 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:28:40.743 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:28:40.743 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.816 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:28:40.817 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:28:40.817 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.854 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:28:40.854 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:28:40.854 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:40.928 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:28:40.928 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:28:40.929 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.001 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:28:41.001 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:28:41.002 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.080 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:28:41.080 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:28:41.081 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:28:41.081 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.104 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:28:41.105 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:28:41.105 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.144 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:28:41.144 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:28:41.145 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:28:41.145 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.156 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:28:41.157 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:28:41.157 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.195 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:28:41.196 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:28:41.196 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:41.247 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:28:41.248 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:28:41.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:28:41.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:41.281 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:28:41.632 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:41.633 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:28:41.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:41.707 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:28:41.782 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:41.782 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:28:41.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:41.859 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:28:42.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:42.087 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:28:42.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:42.187 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:28:42.254 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:42.254 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:28:42.325 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:28:42.327 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/data/mon.a.tgz 2023-11-24T06:28:42.327 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2023-11-24T06:28:42.346 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.4CHSpwL0BF 2023-11-24T06:28:42.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4CHSpwL0BF 2023-11-24T06:28:42.504 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.4CHSpwL0BF 2023-11-24T06:28:42.591 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.4CHSpwL0BF is 483KB 2023-11-24T06:28:42.688 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.4CHSpwL0BF 2023-11-24T06:28:42.706 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/data/mon.b.tgz 2023-11-24T06:28:42.706 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2023-11-24T06:28:42.724 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.OA4mt83nkY 2023-11-24T06:28:42.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OA4mt83nkY 2023-11-24T06:28:42.911 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.OA4mt83nkY 2023-11-24T06:28:42.996 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.OA4mt83nkY is 645KB 2023-11-24T06:28:43.098 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.OA4mt83nkY 2023-11-24T06:28:43.114 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/data/mon.c.tgz 2023-11-24T06:28:43.114 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2023-11-24T06:28:43.170 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.Xc1y7NA81r 2023-11-24T06:28:43.171 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Xc1y7NA81r 2023-11-24T06:28:43.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.Xc1y7NA81r 2023-11-24T06:28:43.434 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.Xc1y7NA81r is 645KB 2023-11-24T06:28:43.536 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.Xc1y7NA81r 2023-11-24T06:28:43.552 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:28:43.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:28:43.555 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:28:43.619 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:28:43.619 DEBUG:teuthology.orchestra.run.smithi114:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:28:43.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:28:43.695 INFO:teuthology.orchestra.run.smithi172.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:28:43.912 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:28:43.918 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:28:43.920 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi114/crash 2023-11-24T06:28:43.921 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:28:43.951 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi172/crash 2023-11-24T06:28:43.952 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:28:43.979 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:28:43.980 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:43.996 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:44.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:44.095 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:44.121 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:28:44.122 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:28:44.138 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:28:44.163 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118872.log 2023-11-24T06:28:44.163 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125422.log 2023-11-24T06:28:44.163 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118826.log 2023-11-24T06:28:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T06:28:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103617.log 2023-11-24T06:28:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110385.log 2023-11-24T06:28:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103617.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.110385.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103617.log.gz 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110385.log.gz 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103564.log 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111134.log 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103474.log 2023-11-24T06:28:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103564.log: /var/log/ceph/ceph-client.1.111134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103337.log 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103564.log.gz 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111134.log.gz 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103474.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103337.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103474.log.gz 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103429.log 2023-11-24T06:28:44.213 INFO:teuthology.orchestra.run.smithi172.stderr: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103337.log.gz 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.109757.log 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.103429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.109749.log 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.109757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103429.log.gz 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.109757.log.gz 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:28:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.109749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112409.log 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.109749.log.gz 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.112409.log: gzip 0.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.1.112409.log.gz-- 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.111724.log 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.111724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111138.log 2023-11-24T06:28:44.215 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111724.log.gz 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.111138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111138.log.gz 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.x.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111728.log 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110389.log 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.111728.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:28:44.216 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111728.log.gz 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.110389.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110389.log.gz 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103382.log 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103382.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103382.log.gz 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103290.log 2023-11-24T06:28:44.217 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.103290.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103290.log.gz 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.b.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103519.log 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.103519.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103519.log.gz 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112413.log 2023-11-24T06:28:44.218 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.112413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112413.log.gz 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118872.log: /var/log/ceph/ceph-client.admin.125422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113979.log 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118872.log.gz 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118826.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125422.log.gz 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127110.log 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6%/var/log/ceph/ceph-client.2.113979.log: -- replaced with /var/log/ceph/ceph-client.admin.118826.log.gz 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr:gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.2.113979.log.gz--verbose 2023-11-24T06:28:44.226 INFO:teuthology.orchestra.run.smithi114.stderr:-- /var/log/ceph/ceph-client.admin.123809.log 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.127110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103533.log 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.127110.log.gz 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118045.log 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.103533.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123809.log.gz 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125936.log 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103533.log.gz 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119531.log 2023-11-24T06:28:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125936.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118045.log.gz 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120717.log 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119531.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125936.log.gz 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119177.log 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120717.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119531.log.gz 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124258.log 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120717.log.gz 2023-11-24T06:28:44.228 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126485.log 2023-11-24T06:28:44.229 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124258.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119177.log.gz 2023-11-24T06:28:44.229 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115558.log 2023-11-24T06:28:44.229 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126485.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124258.log.gz 2023-11-24T06:28:44.229 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115063.log 2023-11-24T06:28:44.229 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126485.log.gz 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112360.log 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119109.log 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115558.log.gz 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112360.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115063.log.gz 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122011.log 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112360.log.gz 2023-11-24T06:28:44.230 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119803.log 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0%/var/log/ceph/ceph-client.admin.122011.log: -- replaced with /var/log/ceph/ceph-client.admin.119109.log.gz 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127266.log 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116163.log 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122011.log.gz 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127266.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119803.log.gz 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129811.log 2023-11-24T06:28:44.231 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127266.log.gz 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116297.log 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129811.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116163.log.gz 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125192.log 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.129811.log.gz 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115604.log 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116297.log.gz 2023-11-24T06:28:44.232 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128511.log 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115604.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125192.log.gz 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125982.log 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115604.log.gz 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117997.log 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128511.log.gz 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128401.log 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117997.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125982.log.gz 2023-11-24T06:28:44.233 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.117997.log.gz 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121597.log 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.128401.log.gz 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121402.log 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109794.log 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121597.log.gz 2023-11-24T06:28:44.234 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111406.log 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121402.log.gz 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.109794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118367.log 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109794.log.gz 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128604.log 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111406.log: /var/log/ceph/ceph-client.admin.118367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116481.log 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111406.log.gz 2023-11-24T06:28:44.235 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117318.log 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128604.log.gz 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116481.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.118367.log.gz 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116025.log 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126715.log 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117318.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116481.log.gz 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115830.log /var/log/ceph/ceph-client.admin.116025.log: 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117318.log.gz 2023-11-24T06:28:44.236 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115420.log 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116025.log.gz 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126715.log.gz 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117550.log 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115830.log: /var/log/ceph/ceph-client.admin.115420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129376.log 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115830.log.gz 2023-11-24T06:28:44.237 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117550.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115420.log.gz 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126877.log 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117550.log.gz 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128558.log 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129376.log: /var/log/ceph/ceph-client.admin.126877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117458.log 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129376.log.gz 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128558.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126877.log.gz 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:28:44.238 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128558.log.gz 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119393.log 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117458.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115979.log 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117458.log.gz 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125238.log 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119393.log.gz 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122516.log 2023-11-24T06:28:44.239 INFO:teuthology.orchestra.run.smithi114.stderr: 86.1%gzip -5 -- replaced with /var/log/ceph/ceph.log.gz--verbose 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr:-- /var/log/ceph/ceph-client.admin.116573.log 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125238.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115979.log.gz 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118918.log 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125238.log.gz 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114971.log 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122516.log.gz 2023-11-24T06:28:44.240 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116573.log.gz 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118091.log 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124989.log 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118918.log.gz 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118091.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114971.log.gz 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127604.log 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118091.log.gz 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123102.log 2023-11-24T06:28:44.241 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124989.log: /var/log/ceph/ceph-client.admin.127604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116117.log 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124989.log.gz 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123102.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127604.log.gz 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111414.log 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123102.log.gz 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116117.log: /var/log/ceph/ceph-client.0.118670.log 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126186.log 2023-11-24T06:28:44.242 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116117.log.gz 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111414.log.gz 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.118670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129489.log 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118670.log.gz 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr:gzip /var/log/ceph/ceph-client.admin.126186.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.126531.log 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124118.log 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126186.log.gz 2023-11-24T06:28:44.243 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126531.log: gzip 88.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.129489.log.gz-- 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127987.log 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126531.log.gz 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125145.log 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128125.log 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124118.log.gz 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125145.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127987.log.gz 2023-11-24T06:28:44.244 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125284.log 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125145.log.gz 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117412.log 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128125.log.gz 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127558.log 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125284.log: /var/log/ceph/ceph-client.admin.117412.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125284.log.gz 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127558.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117412.log.gz 2023-11-24T06:28:44.245 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126301.log 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127558.log.gz 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120579.log 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111416.log 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120579.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126301.log.gz 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118321.log 2023-11-24T06:28:44.246 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.120579.log.gz-5 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr:--verbose -- /var/log/ceph/ceph-client.admin.120440.log 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111416.log: /var/log/ceph/ceph-client.admin.118321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124048.log 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111416.log.gz 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118321.log.gz 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127922.log 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120440.log.gz 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124328.log 2023-11-24T06:28:44.247 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129054.log 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124048.log.gz 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124328.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127922.log.gz 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109954.log 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124328.log.gz 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129054.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129054.log.gz 2023-11-24T06:28:44.248 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118229.log 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.109954.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124489.log 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109954.log.gz 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117272.log 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129192.log 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118229.log.gz 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126140.log 2023-11-24T06:28:44.249 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.124489.log.gz 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111039.log 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117272.log.gz 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110094.log 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129192.log.gz 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128079.log 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126140.log.gz 2023-11-24T06:28:44.250 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110094.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111039.log.gz 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128650.log 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110094.log.gz 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128079.log: /var/log/ceph/ceph-client.admin.128650.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128079.log.gz 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118780.log 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128650.log.gz 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123310.log 2023-11-24T06:28:44.251 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124188.log 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118780.log.gz 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124188.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123310.log.gz 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124188.log.gz 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116803.log 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120763.log 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112130.log 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116803.log.gz 2023-11-24T06:28:44.252 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129238.log 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.120763.log.gz-5 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr:--verbose /var/log/ceph/ceph-client.admin.112130.log: -- /var/log/ceph/ceph-client.admin.112084.log 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129238.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112130.log.gz 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129857.log 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129238.log.gz 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112084.log.gz 2023-11-24T06:28:44.253 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125035.log 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121873.log 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125035.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129857.log.gz 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125035.log.gz 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121873.log.gz 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122784.log 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:28:44.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125081.log 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117098.log 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122784.log.gz 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127312.log 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117098.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125081.log.gz 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127312.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117098.log.gz 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127312.log.gz 2023-11-24T06:28:44.255 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129008.log 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117905.log 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114879.log 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129008.log.gz 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117905.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117905.log.gz 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127650.log 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121965.log 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127650.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114879.log.gz 2023-11-24T06:28:44.256 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.127650.log.gz 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129330.log 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120284.log 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129330.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121965.log.gz 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129330.log.gz 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120284.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120284.log.gz 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110336.log 2023-11-24T06:28:44.257 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116711.log 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 /var/log/ceph/ceph-client.admin.110336.log: --verbose -- /var/log/ceph/ceph-client.admin.110612.log 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116711.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110336.log.gz 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110612.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116711.log.gz 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110612.log.gz 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110842.log 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114741.log 2023-11-24T06:28:44.258 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122562.log 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110842.log.gz 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128447.log 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114741.log.gz 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111045.log 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122562.log.gz 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129627.log 2023-11-24T06:28:44.259 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110750.log 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9%/var/log/ceph/ceph-client.admin.111045.log: -- replaced with /var/log/ceph/ceph-client.admin.128447.log.gz 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129627.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111045.log.gz 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110750.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129627.log.gz 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110750.log.gz 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103664.log 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111040.log 2023-11-24T06:28:44.260 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.103664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118674.log 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103664.log.gz 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111040.log: /var/log/ceph/ceph-client.0.118674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111040.log.gz 0.0% 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.0.118674.log.gz 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120533.log 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111049.log 2023-11-24T06:28:44.261 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116895.log 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120533.log.gz 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111049.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111049.log.gz 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110046.log 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126439.log 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110046.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116895.log.gz 2023-11-24T06:28:44.262 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126439.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110046.log.gz 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126439.log.gz 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116619.log 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123739.log 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116619.log: /var/log/ceph/ceph-client.admin.123739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116619.log.gz 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123739.log.gz 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124419.log 2023-11-24T06:28:44.263 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121689.log 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124419.log: /var/log/ceph/ceph-client.admin.121689.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124419.log.gz 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121689.log.gz 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112222.log 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112222.log: /var/log/ceph/ceph-mds.c.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112222.log.gz 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114464.log 2023-11-24T06:28:44.264 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115155.log 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114464.log: /var/log/ceph/ceph-client.admin.115155.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114464.log.gz 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129719.log 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115155.log.gz 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125797.log 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129719.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129719.log.gz 2023-11-24T06:28:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125797.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125797.log.gz 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119577.log 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126761.log 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119577.log: /var/log/ceph/ceph-client.admin.126761.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119577.log.gz 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126761.log.gz 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121457.log 2023-11-24T06:28:44.266 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129284.log 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121457.log: /var/log/ceph/ceph-client.admin.129284.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121457.log.gz 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129284.log.gz 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120146.log 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116849.log 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120146.log.gz/var/log/ceph/ceph-client.admin.116849.log: 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116849.log.gz 2023-11-24T06:28:44.267 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110288.log 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116071.log 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110288.log: /var/log/ceph/ceph-client.admin.116071.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110288.log.gz 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116071.log.gz 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111669.log 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116389.log 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111669.log: 89.0%/var/log/ceph/ceph-client.admin.116389.log: -- replaced with /var/log/ceph/ceph-client.admin.111669.log.gz 2023-11-24T06:28:44.268 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116389.log.gz 2023-11-24T06:28:44.269 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119849.log 2023-11-24T06:28:44.269 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125889.log 2023-11-24T06:28:44.269 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119849.log: 88.7%/var/log/ceph/ceph-client.admin.125889.log: -- replaced with /var/log/ceph/ceph-client.admin.119849.log.gz 2023-11-24T06:28:44.299 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125889.log.gz 2023-11-24T06:28:44.299 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128309.log 2023-11-24T06:28:44.299 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125843.log 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128309.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128309.log.gz 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125843.log.gz 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123438.log 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109908.log 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123438.log: /var/log/ceph/ceph-client.admin.109908.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123438.log.gz 2023-11-24T06:28:44.300 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109908.log.gz 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120671.log 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110196.log 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120671.log: /var/log/ceph/ceph-client.admin.110196.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120671.log.gz 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110196.log.gz 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110796.log 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115293.log 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110796.log: /var/log/ceph/ceph-client.admin.115293.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110796.log.gz 2023-11-24T06:28:44.301 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115293.log.gz 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113318.log 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119711.log 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.113318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113318.log.gz 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119711.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119711.log.gz 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129422.log 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122173.log 2023-11-24T06:28:44.302 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129422.log: /var/log/ceph/ceph-client.admin.122173.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129422.log.gz 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121549.log 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122173.log.gz 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120008.log 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121549.log: 88.1%/var/log/ceph/ceph-client.admin.120008.log: -- replaced with /var/log/ceph/ceph-client.admin.121549.log.gz 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.120008.log.gz 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126078.log 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110382.log 2023-11-24T06:28:44.303 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126078.log: /var/log/ceph/ceph-client.admin.110382.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126078.log.gz 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110382.log.gz 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111992.log 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120238.log 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111992.log: /var/log/ceph/ceph-client.admin.120238.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111992.log.gz 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120238.log.gz 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129765.log 2023-11-24T06:28:44.304 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117851.log 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129765.log: /var/log/ceph/ceph-client.admin.117851.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129765.log.gz 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124943.log 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117851.log.gz 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121827.log 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124943.log.gz 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128033.log 2023-11-24T06:28:44.305 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121827.log: /var/log/ceph/ceph-client.admin.128033.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121827.log.gz 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128033.log.gz 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123599.log 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123508.log 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123599.log: /var/log/ceph/ceph-client.admin.123508.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123599.log.gz 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123508.log.gz 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120100.log 2023-11-24T06:28:44.306 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120100.log: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120100.log.gz 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109862.log 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116665.log 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.109862.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116665.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109862.log.gz 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111769.log 2023-11-24T06:28:44.307 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116665.log.gz 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114259.log 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124667.log 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114259.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111769.log.gz 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114259.log.gz 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124667.log.gz 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119914.log 2023-11-24T06:28:44.308 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123056.log 2023-11-24T06:28:44.309 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117005.log 2023-11-24T06:28:44.309 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123056.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119914.log.gz 2023-11-24T06:28:44.309 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.123056.log.gz 2023-11-24T06:28:44.309 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119665.log 2023-11-24T06:28:44.309 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114205.log 2023-11-24T06:28:44.310 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117005.log.gz 2023-11-24T06:28:44.310 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119665.log: /var/log/ceph/ceph-client.admin.114205.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119665.log.gz 2023-11-24T06:28:44.310 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114205.log.gz 2023-11-24T06:28:44.310 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110520.log 2023-11-24T06:28:44.310 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122103.log 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129994.log 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122103.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110520.log.gz 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129994.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129994.log.gz 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.122103.log.gz 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115466.log 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120625.log 2023-11-24T06:28:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123194.log 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115466.log.gz 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120625.log: /var/log/ceph/ceph-client.admin.123194.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120625.log.gz 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123194.log.gz 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111417.log 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119285.log 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120817.log 2023-11-24T06:28:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119285.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111417.log.gz 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112682.log 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110566.log 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119285.log.gz 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120817.log: /var/log/ceph/ceph-client.admin.112682.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120817.log.gz 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110566.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110566.log.gz 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.112682.log.gz 2023-11-24T06:28:44.313 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121260.log 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121015.log 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114326.log 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121015.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121260.log.gz 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114326.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121015.log.gz 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114326.log.gz 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125376.log 2023-11-24T06:28:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116435.log 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116343.log 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116435.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125376.log.gz 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116343.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116435.log.gz 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127830.log 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116343.log.gz 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103223.log 2023-11-24T06:28:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111786.log 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127830.log: /var/log/ceph/ceph-client.admin.103223.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127830.log.gz 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111786.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.103223.log.gz 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111786.log.gz 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120330.log 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116757.log 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126807.log 2023-11-24T06:28:44.316 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120330.log.gz 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126807.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116757.log.gz 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.126807.log.gz 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129100.log 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115876.log 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124897.log 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115876.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129100.log.gz 2023-11-24T06:28:44.317 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124897.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115876.log.gz 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124897.log.gz 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113548.log 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121503.log 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.113548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126393.log 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121503.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113548.log.gz 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121503.log.gz 2023-11-24T06:28:44.318 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126393.log.gz 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110242.log 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112544.log 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128916.log 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110242.log.gz 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112544.log: /var/log/ceph/ceph-client.admin.128916.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112544.log.gz 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128916.log.gz 2023-11-24T06:28:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128217.log 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126030.log 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115201.log 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126030.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.128217.log.gz 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115201.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.126030.log.gz 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115201.log.gz 2023-11-24T06:28:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125468.log 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122654.log 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123240.log 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122654.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125468.log.gz 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123240.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122654.log.gz 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123240.log.gz 2023-11-24T06:28:44.321 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129948.log 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111765.log 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.1.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129948.log.gz 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111765.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111765.log.gz 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121735.log 2023-11-24T06:28:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112268.log 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121735.log: /var/log/ceph/ceph-client.admin.112268.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121735.log.gz 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120376.log 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112268.log.gz 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115512.log 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120376.log: /var/log/ceph/ceph-client.admin.115512.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120376.log.gz 2023-11-24T06:28:44.323 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115512.log.gz 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112038.log 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111043.log 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112038.log: /var/log/ceph/ceph-client.admin.111043.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112038.log.gz 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111043.log.gz 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127220.log 2023-11-24T06:28:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109554.log 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127220.log: /var/log/ceph/ceph-client.admin.109554.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.127220.log.gz 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109554.log.gz 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115784.log 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112498.log 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115784.log: /var/log/ceph/ceph-client.admin.112498.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115784.log.gz 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112498.log.gz 2023-11-24T06:28:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125622.log 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125622.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125622.log.gz 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125330.log 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112636.log 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125330.log: /var/log/ceph/ceph-client.admin.112636.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125330.log.gz 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112636.log.gz 2023-11-24T06:28:44.326 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117803.log 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119439.log 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117803.log: /var/log/ceph/ceph-client.admin.119439.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117803.log.gz 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119439.log.gz 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113983.log 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129673.log 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.2.113983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113983.log.gz 2023-11-24T06:28:44.327 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129673.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129673.log.gz 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127466.log 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122738.log 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127466.log: /var/log/ceph/ceph-client.admin.122738.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127466.log.gz 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122738.log.gz 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117188.log 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129535.log 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117188.log: /var/log/ceph/ceph-client.admin.129535.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117188.log.gz 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129535.log.gz 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124759.log 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114556.log 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124759.log: /var/log/ceph/ceph-client.admin.114556.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124759.log.gz 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114556.log.gz 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110474.log 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129904.log 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110474.log: /var/log/ceph/ceph-client.admin.129904.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110474.log.gz 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129904.log.gz 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115247.log 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128696.log 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115247.log: /var/log/ceph/ceph-client.admin.128696.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115247.log.gz 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128696.log.gz 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118183.log 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114649.log 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118183.log: /var/log/ceph/ceph-client.admin.114649.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118183.log.gz 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114649.log.gz 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109663.log 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119049.log 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.109663.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109663.log.gz 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119049.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119049.log.gz 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110142.log 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123148.log 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110142.log: /var/log/ceph/ceph-client.admin.123148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110142.log.gz 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123148.log.gz 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116527.log 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121121.log 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116527.log: /var/log/ceph/ceph-client.admin.121121.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116527.log.gz 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121121.log.gz 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117642.log 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120192.log 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117642.log: /var/log/ceph/ceph-client.admin.120192.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117642.log.gz 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120192.log.gz 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117364.log 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112590.log 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117364.log: /var/log/ceph/ceph-client.admin.112590.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117364.log.gz 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112590.log.gz 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112406.log 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118137.log 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112406.log: /var/log/ceph/ceph-client.admin.118137.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112406.log.gz 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118137.log.gz 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111668.log 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115692.log 2023-11-24T06:28:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111668.log: /var/log/ceph/ceph-client.admin.115692.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111668.log.gz 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115692.log.gz 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111048.log 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114925.log 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111048.log: /var/log/ceph/ceph-client.admin.114925.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111048.log.gz 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114925.log.gz 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114833.log 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117951.log 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114833.log: /var/log/ceph/ceph-client.admin.117951.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114833.log.gz 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117951.log.gz 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128962.log 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114418.log 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128962.log: /var/log/ceph/ceph-client.admin.114418.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128962.log.gz 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114418.log.gz 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123669.log 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123010.log 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123669.log: /var/log/ceph/ceph-client.admin.123010.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123669.log.gz 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.123010.log.gz 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114602.log 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110704.log 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114602.log: /var/log/ceph/ceph-client.admin.110704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114602.log.gz 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110704.log.gz 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125712.log 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128171.log 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125712.log: /var/log/ceph/ceph-client.admin.128171.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125712.log.gz 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128171.log.gz 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110658.log 2023-11-24T06:28:44.339 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:28:44.344 INFO:teuthology.orchestra.run.smithi172.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:28:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110658.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110658.log.gz 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96932.log 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.tmp-client.admin.96932.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96932.log.gz 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121643.log 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121643.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121643.log.gz 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117734.log 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117734.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117734.log.gz 2023-11-24T06:28:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112452.log 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112452.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112452.log.gz 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117596.log 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117596.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117596.log.gz 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126347.log 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126347.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126347.log.gz 2023-11-24T06:28:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111756.log 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111756.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111756.log.gz 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126623.log 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126623.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126623.log.gz 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116251.log 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116251.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116251.log.gz 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124851.log 2023-11-24T06:28:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124851.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.124851.log.gz 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122057.log 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122057.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122057.log.gz 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129581.log 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129581.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129581.log.gz 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117052.log 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117052.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117052.log.gz 2023-11-24T06:28:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127876.log 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127876.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127876.log.gz 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117504.log 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117504.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117504.log.gz 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114510.log 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114510.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114510.log.gz 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111415.log 2023-11-24T06:28:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111415.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111415.log.gz 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111413.log 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111413.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111413.log.gz 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122922.log 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109458.log 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122922.log: /var/log/ceph/ceph-client.admin.109458.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122922.log.gz 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109458.log.gz 2023-11-24T06:28:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114372.log 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114372.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114372.log.gz 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126669.log 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126669.log.gz 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115109.log 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115109.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115109.log.gz 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122608.log 2023-11-24T06:28:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122608.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122608.log.gz 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128263.log 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128263.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128263.log.gz 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123879.log 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123879.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123879.log.gz 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127512.log 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127512.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127512.log.gz 2023-11-24T06:28:44.359 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125514.log 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.125514.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125514.log.gz 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126232.log 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126232.log.gz 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118275.log 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118275.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118275.log.gz 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115738.log 2023-11-24T06:28:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115738.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115738.log.gz 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114787.log 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114787.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114787.log.gz 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110000.log 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110000.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.110000.log.gz 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127738.log 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127738.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127738.log.gz 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111044.log 2023-11-24T06:28:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111044.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111044.log.gz 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121213.log 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121213.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121213.log.gz 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121167.log 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121167.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121167.log.gz 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119961.log 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119961.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119961.log.gz 2023-11-24T06:28:44.362 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124713.log 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124713.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124713.log.gz 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120925.log 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120925.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120925.log.gz 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122406.log 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.122406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122406.log.gz 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120054.log 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120054.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120054.log.gz 2023-11-24T06:28:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118437.log 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.118437.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118437.log.gz 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122410.log 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111670.log 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.122410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122410.log.gz 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111670.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111670.log.gz 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121306.log 2023-11-24T06:28:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121306.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121306.log.gz 2023-11-24T06:28:44.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:28:44.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117688.log 2023-11-24T06:28:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.117688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117688.log.gz 2023-11-24T06:28:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112176.log 2023-11-24T06:28:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112314.log 2023-11-24T06:28:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112176.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112176.log.gz 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112314.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112314.log.gz 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127784.log 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127784.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127784.log.gz 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115017.log 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.115017.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115017.log.gz 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116941.log 2023-11-24T06:28:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.116941.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116941.log.gz 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111758.log 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111758.log.gz 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112836.log 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.112836.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112836.log.gz 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127358.log 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.127358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127358.log.gz 2023-11-24T06:28:44.370 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113326.log 2023-11-24T06:28:44.371 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.113326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113326.log.gz 2023-11-24T06:28:44.371 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121354.log 2023-11-24T06:28:44.371 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121354.log.gz 2023-11-24T06:28:44.371 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124621.log 2023-11-24T06:28:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124621.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124621.log.gz 2023-11-24T06:28:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126577.log 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.126577.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126577.log.gz 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121781.log 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121781.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121781.log.gz 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123368.log 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123368.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123368.log.gz 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110428.log 2023-11-24T06:28:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.110428.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110428.log.gz 2023-11-24T06:28:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119485.log 2023-11-24T06:28:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119485.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119485.log.gz 2023-11-24T06:28:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109748.log 2023-11-24T06:28:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.109748.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109748.log.gz 2023-11-24T06:28:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:28:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119757.log 2023-11-24T06:28:44.391 INFO:teuthology.orchestra.run.smithi172.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T06:28:44.398 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.119757.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.119757.log.gz 2023-11-24T06:28:44.400 INFO:teuthology.orchestra.run.smithi114.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:28:44.403 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122876.log 2023-11-24T06:28:44.404 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122876.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122876.log.gz 2023-11-24T06:28:44.404 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128355.log 2023-11-24T06:28:44.405 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121919.log 2023-11-24T06:28:44.405 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.128355.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128355.log.gz 2023-11-24T06:28:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.121919.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121919.log.gz 2023-11-24T06:28:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111411.log 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111411.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111411.log.gz 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114695.log 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.114695.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114695.log.gz 2023-11-24T06:28:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120487.log 2023-11-24T06:28:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.120487.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120487.log.gz 2023-11-24T06:28:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127114.log 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.127114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.127114.log.gz 2023-11-24T06:28:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129146.log 2023-11-24T06:28:44.410 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.129146.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129146.log.gz 2023-11-24T06:28:44.410 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122830.log 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.122830.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122830.log.gz 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111401.log 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111401.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111401.log.gz 2023-11-24T06:28:44.412 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124805.log 2023-11-24T06:28:44.414 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.124805.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.124805.log.gz 2023-11-24T06:28:44.417 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111047.log 2023-11-24T06:28:44.417 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.111047.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111047.log.gz 2023-11-24T06:28:44.418 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123978.log 2023-11-24T06:28:44.418 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.123978.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123978.log.gz 2023-11-24T06:28:44.419 INFO:teuthology.orchestra.run.smithi114.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:28:44.425 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:28:44.435 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:28:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:28:44.479 INFO:teuthology.orchestra.run.smithi172.stderr: 2023-11-24T06:28:44.479 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m0.300s 2023-11-24T06:28:44.479 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m1.192s 2023-11-24T06:28:44.479 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.068s 2023-11-24T06:28:44.479 INFO:teuthology.orchestra.run.smithi114.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T06:28:44.482 INFO:teuthology.orchestra.run.smithi114.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:28:44.501 INFO:teuthology.orchestra.run.smithi114.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:28:44.515 INFO:teuthology.orchestra.run.smithi114.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:28:44.649 INFO:teuthology.orchestra.run.smithi114.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:28:44.651 INFO:teuthology.orchestra.run.smithi114.stderr: 2023-11-24T06:28:44.651 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m0.499s 2023-11-24T06:28:44.651 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m1.863s 2023-11-24T06:28:44.652 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m0.241s 2023-11-24T06:28:44.652 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:28:44.653 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi114/log 2023-11-24T06:28:44.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:28:45.261 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi172/log 2023-11-24T06:28:45.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:28:45.480 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:28:45.501 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:28:45.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:28:45.504 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:28:45.581 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:28:45.590 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:28:45.591 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:28:45.593 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:28:45.607 INFO:teuthology.orchestra.run.smithi114.stderr:bash: ntpq: command not found 2023-11-24T06:28:45.610 INFO:teuthology.orchestra.run.smithi172.stderr:bash: ntpq: command not found 2023-11-24T06:28:45.611 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:28:45.611 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2023-11-24T06:28:45.611 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +1752us[+1753us] +/- 52ms 2023-11-24T06:28:45.611 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -2092us[-2091us] +/- 72ms 2023-11-24T06:28:45.612 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -442us[ -442us] +/- 70ms 2023-11-24T06:28:45.612 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:28:45.614 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:28:45.614 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2023-11-24T06:28:45.614 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +1787us[+1780us] +/- 52ms 2023-11-24T06:28:45.614 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -2050us[-2050us] +/- 72ms 2023-11-24T06:28:45.615 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 -407us[ -407us] +/- 70ms 2023-11-24T06:28:45.615 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:28:45.616 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:28:45.625 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:28:45.625 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:28:45.634 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:28:45.752 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:28:45.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:28:45.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:45.884 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:28:45.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:45.917 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:28:45.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:28:45.938 INFO:teuthology.task.internal:Duration was 1000.214586 seconds 2023-11-24T06:28:45.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:28:45.948 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:28:45.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:28:45.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:28:45.995 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:28:46.003 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:28:46.215 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:28:46.215 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2023-11-24T06:28:46.215 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:28:46.237 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2023-11-24T06:28:46.237 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:28:46.259 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:28:46.259 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:28:46.279 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:28:46.339 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:28:46.339 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:28:46.364 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:28:46.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:28:46.465 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:28:46.465 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:28:46.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:28:46.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:28:46.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:28:46.539 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:28:46.562 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2023-11-24T06:28:46.587 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2023-11-24T06:28:46.601 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:28:46.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:46.633 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:28:46.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:46.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:28:46.669 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:28:46.670 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi114 2023-11-24T06:28:46.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:28:46.713 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559/remote/smithi172 2023-11-24T06:28:46.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:28:46.751 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:28:46.751 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:28:46.756 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:28:46.801 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:28:46.814 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:28:46.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:28:46.828 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:28:46.828 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:28:46.831 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:28:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:28 /home/ubuntu/cephtest 2023-11-24T06:28:46.858 INFO:teuthology.orchestra.run.smithi172.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:28 /home/ubuntu/cephtest 2023-11-24T06:28:46.860 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:28:46.871 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:28:46.899 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:28:46.928 DEBUG:teuthology.nuke:shortname: smithi114 2023-11-24T06:28:46.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:28:46.955 DEBUG:teuthology.nuke:shortname: smithi172 2023-11-24T06:28:46.956 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:28:46.960 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:06:23.689373', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:28:46.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:06:23.687360', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:28:47.014 INFO:teuthology.orchestra.console.smithi114:Power off 2023-11-24T06:28:47.015 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:28:47.039 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2023-11-24T06:28:47.040 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:47.058 INFO:teuthology.orchestra.console.smithi172:Power off 2023-11-24T06:28:47.059 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:28:47.071 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-11-24T06:28:47.087 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2023-11-24T06:28:47.088 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:47.116 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-11-24T06:28:51.073 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:51.117 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:51.200 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-11-24T06:28:51.242 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-11-24T06:28:55.200 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:55.243 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:55.311 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-11-24T06:28:55.352 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2023-11-24T06:28:59.313 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:59.353 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:28:59.425 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-11-24T06:28:59.463 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2023-11-24T06:28:59.527 INFO:teuthology.orchestra.console.smithi114:Power off completed 2023-11-24T06:28:59.564 INFO:teuthology.orchestra.console.smithi172:Power off completed 2023-11-24T06:28:59.672 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1000.2145862579346 failure_reason: 'Test failure: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3617ea8c90484640870736dfa88f7af8 status: fail success: false 2023-11-24T06:28:59.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:28:59.752 INFO:teuthology.run:FAIL